Skip to main content

Setting up a dev environment

protohaven_api

Clone the repo and install dependencies

To setup development of protohaven_api repository, you'll need git, python 3 (with pip), and nodejs installed:

# Clone the repository
git clone git@github.com:protohaven/protohaven_api.git

# Install dependencies
cd protohaven_api
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Set up pre-commit validation hooks
pip install pre-commit
pre-commit install

Set up NocoDB

We use NocoDB as a local/dev database replacement for the production Airtable DB. It's easy to run via docker with these instructions. Follow the instructions to getdownload a local instance of NocoDB running:- DO NOT START THE CONTAINER YET:

git clone --depth 1 https://github.com/nocodb/nocodb
cd nocodb/docker-compose/2_pg
docker compose up -d

This should be running by default at http://localhost:8080.

Next, download the following backup files and place them in nocodb/docker-compose/2_pg:

Then run these commands to populate data (adapted from this tutorial):

# Start and load the postgres data - two different terminals
docker compose up root_db
cat dump.sql | docker exec -i 2_pg-root_db-1 psql -U postgres -d root_db

# Load the other nocodb data into the main volume
docker run --rm --volume 2_pg_nc_data:/usr/app/data -v $(pwd):/backup ubuntu bash -c "cd /usr/app/data && tar xvf /backup/nc_data.tar --strip 1

Finally, start the container:

docker compose up -d

Wait for initialization to settle, then browse to http://localhost:8080. You can login with

When running the dev environment, you'll want to ensure NocoDB is running so that it can be reached for queries.

Set up other mock data sources

Download mock_data.pkl to have some sanitized test data to operate on. Place this file in the root of the repository.

Run the CLI

# Test your setup - you should see a YAML formatted message about proposed classes 
python3 -m protohaven_api.cli class_proposals

Run Tests

To run all python tests: python -m pytest -vv, optionally with -k test_substring to run a specific test.

Run API Flask server / Svelte frontend 

# In one console, run this to start a "minimal" Flask server without enforced access control:
cd protohaven_api
source venv/bin/activate
MQTT=false CORS=true UNSAFE_NO_RBAC=true flask --app protohaven_api.main run

# In another console, run this to start a hot-reloading Svelte dev instance:
cd protohaven_api/svelte
npm install
npm run dev

Run Bookstack

Download env.secret and place it in protohaven_api/bookstack/.

We run Bookstack in a Docker container with several volumes/files mounted - but Bookstack's first-run initialization process doesn't kick off if files are already present. 

To run Bookstack for the first time, edit the protohaven_api/bookstack/docker-compose.yaml file and comment out the lines noted with READ THIS IF STARTING FROM SCRATCH.

Then run:

cd protohaven_api/bookstack
docker compose up

and wait for the initialization to settle, before continuing the listed steps in the YAML comments. 

Ctrl+C to take the containers down, then revert your changes to the YAML file and rerun docker compose up.

To view debug logs:

 tail -f protohaven_api/bookstack/bookstack_app_data/log/bookstack/laravel.log
"Invalid redirect_uri parameter value."

Because it's a local instance trying to do OAuth authentication with Neon CRM, you may see this login error when initially trying to view a page. The trick to get around this is:

Note: this probably still requires prod Neon credentials and should be adjusted to allow login if a dev environment var (e.g. APP_URL with localhost) is detected.

  1. Open a new tab, and open the console so you can see network requests
  2. Browse to wiki.protohaven.org - logout if you're already logged in. You should be taken to a Neon login page
  3. Submit your login credentials and get redirected back to the prod wiki.
  4. Search for /login?code=<random string>
  5. Replace the base of the URL with http://localhost:6875 and browse to it to login to the local instance

Run Wordpress

To start a wordpress dev container, run the following:

cd protohaven_api/wordpress
docker compose up

Note that the plugin directories are automatically installed via volume mounts in docker-compose.yaml, however to get live updates you'll also need to run the nodejs script for the plugin you're currently editing:

cd protohaven_api/wordpress/protohaven-class-ticker
npm install
npm run start

Then you can browse to http://localhost:8080 to interact with your wordpress server.

Snapshotting dev state

Nocodb

With Nocodb running, use these commands to snapshot state into two shareable files:

docker run --rm --volumes-from 2_pg-nocodb-1 -v $(pwd):/backup ubuntu tar cvf /backup/nc_data.tar /usr/app/data
docker exec -it 2_pg-root_db-1 pg_dump -U postgres -d root_db > dump.sql

Back to top