Skip to main content

Setting up a dev environment

protohaven_api

Clone the repo and install dependencies

To setup development of protohaven_api repository, you'll need git, python 3 (with pip), and nodejs installed:

# Clone the repository
git clone git@github.com:protohaven/protohaven_api.git

# Install dependencies
cd protohaven_api
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Set up pre-commit validation hooks
pip install pre-commit
pre-commit install

Set up NocoDB

We use NocoDB as a local/dev database replacement for the production Airtable DB. It's easy to run via docker with these instructions. Follow the instructions to download a local instance of NocoDB - DO NOT START THE CONTAINER YET:

git clone --depth 1 https://github.com/nocodb/nocodb
cd nocodb/docker-compose/2_pg

Next, download the following backup files and place them in nocodb/docker-compose/2_pg:

Then run these commands to populate data (adapted from this tutorial):

This next step requires Docker to run NocoDB - make sure this is installed first before continuing.

# Start and load the postgres data, before Nocodb has a chance to initialize
docker compose up -d root_db
cat dump.sql | docker exec -i 2_pg-root_db-1 psql -U postgres -d root_db

# Load the other nocodb data into the volume before nocodb inits it
docker run --rm --volume 2_pg_nc_data:/usr/app/data -v $(pwd):/backup ubuntu bash -c "cd /usr/app/data && tar xvf /backup/nc_data.tar --strip 1"

Finally, start the container:

docker compose down && docker compose up -d

Wait for initialization to settle, then browse to http://localhost:8080. You can login with

When running the protohaven_api flask server in your dev environment, you'll want to ensure NocoDB is running so that it can be reached for queries.

Set up other mock data sources

Download mock_data.pkl to have some sanitized test data to operate on. Place this file in the root of the repository.

Run the CLI

# Test your setup - you should see a YAML formatted message about proposed classes 
python3 -m protohaven_api.cli class_proposals

Run Tests

To run all python tests: python -m pytest -vv, optionally with -k test_substring to run a specific test.

Run API Flask server / Svelte frontend 

# In one console, run this to start a "minimal" Flask server without enforced access control:
cd protohaven_api
source venv/bin/activate
MQTT=false CORS=true UNSAFE_NO_RBAC=true flask --app protohaven_api.main run

# In another console, run this to start a hot-reloading Svelte dev instance:
cd protohaven_api/svelte
npm install
npm run dev

Run Bookstack

Download env.secret and place it in protohaven_api/bookstack/.

We run Bookstack in a Docker container with several volumes/files mounted - but Bookstack's first-run initialization process doesn't kick off if files are already present. 

To run Bookstack for the first time, edit the protohaven_api/bookstack/docker-compose.yaml file and comment out the lines noted with READ THIS IF STARTING FROM SCRATCH.

Then run:

cd protohaven_api/bookstack
docker compose up

and wait for the initialization to settle, before continuing the listed steps in the YAML comments. 

Ctrl+C to take the containers down, then revert your changes to the YAML file and rerun docker compose up.

To view debug logs:

 tail -f protohaven_api/bookstack/bookstack_app_data/log/bookstack/laravel.log
"Invalid redirect_uri parameter value."

Because it's a local instance trying to do OAuth authentication with Neon CRM, you may see this login error when initially trying to view a page. The trick to get around this is:

Note: this probably still requires prod Neon credentials and should be adjusted to allow login if a dev environment var (e.g. APP_URL with localhost) is detected.

  1. Open a new tab, and open the console so you can see network requests
  2. Browse to wiki.protohaven.org - logout if you're already logged in. You should be taken to a Neon login page
  3. Submit your login credentials and get redirected back to the prod wiki.
  4. Search for /login?code=<random string>
  5. Replace the base of the URL with http://localhost:6875 and browse to it to login to the local instance

Run Wordpress

To start a wordpress dev container, run the following:

cd protohaven_api/wordpress
docker compose up

Note that the plugin directories are automatically installed via volume mounts in docker-compose.yaml, however to get live updates you'll also need to run the nodejs script for the plugin you're currently editing:

cd protohaven_api/wordpress/protohaven-class-ticker
npm install
npm run start

Then you can browse to http://localhost:8080 to interact with your wordpress server.

Snapshotting dev state

Nocodb

With Nocodb running, use these commands to snapshot state into two shareable files:

docker run --rm --volumes-from 2_pg-nocodb-1 -v $(pwd):/backup ubuntu tar cvf /backup/nc_data.tar /usr/app/data
docker exec -it 2_pg-root_db-1 pg_dump -U postgres -d root_db > dump.sql

Back to top