Skip to main content

Setting up a dev environment

protohaven_api

Clone the repo and install dependencies

To setup development of protohaven_api repository, you'll need git, python 3 (with pip), and nodejs installed:

# Clone the repository
git clone git@github.com:protohaven/protohaven_api.git

# Install dependencies
cd protohaven_api
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Set up pre-commit validation hooks
pip install pre-commit
pre-commit install

Set up NocoDB

We use NocoDB as a local/dev database replacement for the production Airtable DB. It's easy to run via docker with these instructions. Follow the instructions to download a local instance of NocoDB - DO NOT START THE CONTAINER YET:

git clone --depth 1 https://github.com/nocodb/nocodb
cd nocodb/docker-compose/2_pg

Next, download the following backup files and place them in nocodb/docker-compose/2_pg:

nc_data.tar

dump.sql

Then run these commands to populate data (adapted from this tutorial):

This next step requires Docker to run NocoDB - make sure this is installed first before continuing.

# If you already have a running instance of Nocodb and wish to re-initialize, 
# first we must stop the containers and delete the volumes:
docker compose down
docker volume rm 2_pg_db_data 2_pg_nc_data

# Start and load the postgres data, before Nocodb has a chance to initialize
docker compose up -d root_db
cat dump.sql | docker exec -i 2_pg-root_db-1 psql -U postgres -d root_db

# Load the other nocodb data into the volume before nocodb inits it
docker run --rm --volume 2_pg_nc_data:/usr/app/data -v $(pwd):/backup ubuntu bash -c "cd /usr/app/data && tar xvf /backup/nc_data.tar --strip 1"

Finally, start the container:

docker compose down && docker compose up -d

Wait for initialization to settle, then browse to http://localhost:8080. You can login with

When running the protohaven_api flask server in your dev environment, you'll want to ensure NocoDB is running so that it can be reached for queries.

Set up other mock data sources

Download mock_data.pkl to have some sanitized test data to operate on. Place this file in the root of the repository.

Run the CLI

# Test your setup - you should see a YAML formatted message about proposed classes 
python3 -m protohaven_api.cli class_proposals

Run Tests

To run all python tests: python -m pytest -vv, optionally with -k test_substring to run a specific test.

Run API Flask server / Svelte frontend 

# In one console, run this to start a "minimal" Flask server without enforced access control:
cd protohaven_api
source venv/bin/activate
MQTT=false CORS=true UNSAFE_NO_RBAC=true flask --app protohaven_api.main --debug run

# In another console, run this to start a hot-reloading Svelte dev instance:
cd protohaven_api/svelte
npm install
npm run dev

Logging in as different users

There's a helpful handler for setting your login session to one of the accounts in the "Fake Neon" base of your local Nocodb instance:

https://localhost:5173/admin/login_as?neon_id=1797 

Replace 1797 with an account ID of your choice to populate your browser session with info from the matching row in Fake Neon > accounts. 1791 in this case is set up as an instructor so can be useful for working on the class scheduling UI and automation.

Run Bookstack (http://localhost:6875)

First time setup

Download env.secret and db_env.secret and place them in protohaven_api/bookstack/.

We run Bookstack in a Docker container with several volumes/files mounted - but Bookstack's first-run initialization process doesn't kick off if files are already present. To run Bookstack for the first time so this initialization happens:

cd protohaven_api/bookstack
docker compose --profile no_plugin up

Wait for initialization to settle (a couple of minutes), then stop the service by hitting Ctrl+C

Adding custom head content

Manually copy the contents of custom_head_content.html into the "Custom HTML Head Content" textarea in the settings of the wiki.

Normal development

To start a server after the first run:

cd protohaven_api/bookstack
docker compose --profile protohaven up -d

To view debug logs:

 tail -f protohaven_api/bookstack/bookstack_app_data/log/bookstack/laravel.log
"Invalid redirect_uri parameter value."

Because it's a local instance trying to do OAuth authentication with Neon CRM, you may see this login error when initially trying to view a page.

When the server is running locally, authentication can be shortcut using http://localhost:6875/login_as?email=admin@admin.com. You may also create other users and login as them by changing the value of the email param.

Testing OAuth and role syncing

OAuth testing requires production credentials. This should be done on the Cron server in the ~/protohaven_api instance of the repository.

With the a running instance of Bookstack on the Cron server:

  1. Open a new tab, and open the browser console to the Networking tab so you can see network requests.
  2. Browse to wiki.protohaven.org - logout if you're already logged in, using the triple-dot menu on the top right of the page. You should be immediately taken to a Neon login page.
  3. Submit your login credentials and get redirected back to the prod wiki.
  4. Search for /login?code=<random string>
  5. Replace the base of the URL with the URL of the bookstack instance and browse to it to login to the local instance.

Run Wordpress

To start a wordpress dev container, run the following:

cd protohaven_api/wordpress
docker compose up

Note that the plugin directories are automatically installed via volume mounts in docker-compose.yaml, however to get live updates you'll also need to run the nodejs script for the plugin you're currently editing:

cd protohaven_api/wordpress/protohaven-class-ticker
npm install
npm run start

Then you can browse to http://localhost:8080 to interact with your wordpress server.

Snapshotting dev state

Nocodb

With Nocodb running, use these commands to snapshot state into two shareable files:

docker run --rm --volumes-from 2_pg-nocodb-1 -v $(pwd):/backup ubuntu tar cvf /backup/nc_data.tar /usr/app/data
docker exec -it 2_pg-root_db-1 pg_dump -U postgres -d root_db > dump.sql

Back to top