Remove libpqtypes, use plain libpq - #95
Draft
arybczak wants to merge 7 commits into
Draft
Conversation
arybczak
changed the title
arybczak
force-pushed
the
no-libpqtypes
branch
7 times, most recently
from
July 12, 2026 01:05
ed40028 to
b5aaed1
Compare
arybczak
force-pushed
the
no-libpqtypes
branch
3 times, most recently
from
July 12, 2026 03:41
738b5bd to
0f44dcf
Compare
This was referenced Jul 12, 2026
arybczak
force-pushed
the
no-libpqtypes
branch
9 times, most recently
from
July 20, 2026 06:56
d2fb50f to
aa47fec
Compare
arybczak
changed the title
The vendored libpqtypes C library is gone. Queries are executed with the asynchronous API of plain libpq and the binary transport format is handled on the Haskell side: per-type format strings are replaced by type Oids, fromSQL becomes a monadic RowDecoder consuming consecutive fields of a result, and encoding of composite types along with the Composite machinery is dropped in favour of decoding combinators. See CHANGELOG.md for the full list of user visible changes.
The encoders and decoders of the binary wire format now live in Internal.Encoding and Internal.Decoding, derived from the postgresql-binary package (MIT licensed, attribution added to LICENSE) and adjusted where its behavior didn't suit us. The divergences are noted at their definitions; the one that prompted this is that the text encoders silently drop NUL characters, which turns a precise server side error into unreported data corruption. The Range type is vendored along with them, as it's part of the API of the range codecs.
Map inet to (IP, Int) internally so that the netmask is carried alongside the address, which lets FromSQL IP reject values with host bits set instead of silently dropping them, and cidr to IPRange.
Assorted follow-ups to the rewrite: - fetchMaybe and fetchOne check the number of delivered rows before decoding any of them, so a result of the wrong size is reported as such rather than as whatever decoding its first row happens to produce, and fetchOne says that it expected exactly one row. - FromSQL instances for Int and Word are defined with TypeError, pointing at Int64 and Word64: their size is architecture dependent, so decoding them is not reliable. - Errors thrown from the on demand connection mode are no longer masked by the failure of the ROLLBACK ending the automatic transaction. - Interval gets a structural roundtrip test, PQFormat a note about newtype deriving not working when the Oid differs, and the haddocks are free of warnings.
How an array of a type is decoded becomes a FromSQL method, fromSQLArray (with fromSQLList backing the list instance on top of it), so that an instance can supply something faster than the default decodeArray fromSQL. Scalar types override it with decodeScalarArray, which decodes the elements with a value decoder directly instead of running a full RowDecoder per element, and checks their type once for the whole array. Errors of the elements keep reporting their position as the column of a ConversionError, the way the elements of decodeArray do.
Cover both that the execution of a query (plain or prepared) can be interrupted and that a query interrupted while it is still being sent is cut short rather than waited out, which is what requires the pending output to be flushed before cancellation is requested.
A single request is not enough: the server discards one that arrives before the backend started executing the query, as cancellation is only acted upon while a command is running. Nothing reports this — the request is acknowledged as received either way and a discarded one leaves no trace at all — so the only sign that one took effect is the query ending early. An interrupted thread could therefore end up waiting for the query to run to completion, which in particular defeated timeouts. The request is now repeated until the results show up, with the wait between repetitions bounded and the delay doubling from 50ms up to 400ms. The common case, where the exception arrives while the query is genuinely executing, is unaffected: the first request lands, the results become available within a round trip and nothing further is sent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Drop the bundled libpqtypes C library. Queries are now executed with the asynchronous API of plain libpq and the binary transport format is handled by the code based on postgresql-binary package (with some bugs fixed).
The library now supports decoding of arbitrary composite (row) types (in particular anonymous ones), named composite types are no longer necessary (and thus no longer have to be registered up-front).
Support for enum types is also improved, single values don't need casts anymore when passed as parameters (arrays of enums still do unfortunately).
Benchmark results:
masterno-libpqtypes