ntx-builder: move to sqlite framework#2353
Conversation
There was a problem hiding this comment.
Some thoughts, leaving inline so we can discuss more easily.
First some organizational ones:
- I think we should wrap the db crate's read and write tx again here, and then expose the queries / writes as methods instead of free functions.
- I would prefer separate folder per sqlite query i.e. not bundling multiple per file, and no
sqlfolder.
More important though:
- Everywhere still has access to the
Databasestruct.- This means everywhere can still write if they so choose.
- I think the
SQLiteframework should not give back theDatabasestruct on construction.- Instead return:
(WriteTx, Pool<ReadTx>). - That way we cannot mix and match.
- Instead return:
There was a problem hiding this comment.
I implemented the wrapper struct (NtxDb) and replace the read/writes with methods, and did that folder organization that you mention. It looks cleaner now.
Regarding the other comments:
Everywhere still has access to the Database struct.
With the wrapper this was fixed, since all read/writes happens through the methods.
I think the SQLite framework should not give back the Database struct on construction.
I don't think I fully understand, so I want to clarify before changing things. Essentially in the ntx-builder we have the block subscription, the server, and the actor context. And only the block stream one (main loop) needs write access, so what I should do is to have 2 wrapper structs, one that only implements reading methods, and another with write capacity?
There was a problem hiding this comment.
What I'm getting at, is that it should not be possible to create more than one write connection. And in addition, we want to enforce that writing is unavailable to read-only paths.
For example, in the node, the RPC module should have zero access to database writing. At present it would own a Database and can invoke write at any time. And in the ntx-builder, the actors should have zero access to writing.
What I'm suggesting is that once a database is configured, you not longer need Database, and you only care about the connection pool for reads and the single write connection.
One way of modelling this is:
// A single `WriteConnection`, and a connection pool returning only `ReadOnlyConnection`.
let (db_writer, db_read_pool) = Database::new();
// In ntxb, the writer is owned by the central coordinator, and
// the actors get only the read pool.There was a problem hiding this comment.
I addressed that in df523c1 . Though, and regarding previous comments, when I introduced the framework and migrated the validator component to it I used free functions for the queries instead of the wrapper struct + methods that I used in this PR. I will open a another PR migrating the validator to that design instead of doing that here.
closes #2250
Summary
Migrates the ntx-builder database layer off Diesel and onto the shared
miden-node-db SQLite framework.Why:
How:
Dieselquery modules with plain query functions overReadTx/WriteTxand hand-written SQL indb/sql/.001_initial.sqlschema), so existing databases stay compatible.Changelog