Storage

HARP needs somewhere to keep the traffic it records, and it lets you match that storage to your environment instead of forcing one choice on you.

By default, with no configuration, HARP uses SQLite, so you can try it without standing up a database. It is convenient for development and testing, but not for production, where its concurrency and performance limits show.

In production you point HARP at datastores you already run rather than operating another one. It separates two layers:

• Core storage keeps transactions, messages, users, tags and flags in a SQL database through SQLAlchemy. SQLite is the default for development, PostgreSQL is recommended for production, and MySQL and MariaDB are also tested.
• Blob storage holds binary message content such as headers and bodies. It is content-addressable, so a payload repeated across thousands of transactions is stored only once, saving space. By default it reuses the SQL database, and Redis is the recommended backend for production.