sql_log_normalizer.rb

lib/sequel/extensions/sql_log_normalizer.rb
Last Update: 2021-10-11 08:09:16 -0700

The sql_log_normalizer extension normalizes the SQL that is logged, removing the literal strings and numbers in the SQL, and removing the logging of any bound variables:

ds = DB[:table].first(a: 1, b: 'something')
# Without sql_log_normalizer extension
# SELECT * FROM "table" WHERE (("a" = 1) AND ("b" = 'something')) LIMIT 1

# With sql_log_normalizer_extension
# SELECT * FROM "table" WHERE (("a" = ?) AND ("b" = ?)) LIMIT ?

The normalization is done by scanning the SQL string being executed for literal strings and numbers, and replacing them with question marks. While this should work for all or almost all production queries, there are pathlogical queries that will not be handled correctly, such as the use of apostrophes in identifiers:

DB[:"asf'bar"].where(a: 1, b: 'something').first
# Logged as:
# SELECT * FROM "asf?something')) LIMIT ?

The expected use case for this extension is when you want to normalize logs to group similar queries, or when you want to protect sensitive data from being stored in the logs.

Related module: Sequel::SQLLogNormalizer