logging URI query params with lograge – Bibliographic Wilderness Skip to content Bibliographic Wilderness Menu About Contact logging URI query params with lograge jrochkind General August 4, 2021August 5, 2021 The lograge gem for taming Rails logs by default will lot the path component of the URI, but leave out the query string/query params. For instance, perhaps you have a URL to your app /search?q=libraries. lograge will log something like: method=GET path=/search format=html… The q=libraries part is completely left out of the log. I kinda want that part, it’s important. The lograge README provides instructions for “logging request parameters”, by way of the params hash. I’m going to modify them a bit slightly to: use the more recent custom_payload config instead of custom_options. (I’m not certain why there are both, but I think mostly for legacy reasons and newer custom_payload? is what you should read for?) If we just put params in there, then a bunch of ugly "foo"} OK. The params hash isn’t exactly the same as the query string, it can include things not in the URL query string (like controller and action, that we have to strip above, among others), and it can in some cases omit things that are in the query string. It just depends on your routing and other configuration and logic. The params hash itself is what default rails logs… but what if we just log the actual URL query string instead? Benefits: it’s easier to search the logs for actually an exact specific known URL (which can get more complicated like /search?q=foo&range%5Byear_facet_isim%5D%5Bbegin%5D=4&source=foo or something). Which is something I sometimes want to do, say I got a URL reported from an error tracking service and now I want to find that exact line in the log. I actually like having the exact actual URL (well, starting from path) in the logs. It’s a lot simpler, we don’t need to filter out controller/action/format/id etc. It’s actually a bit more concise? And part of what I’m dealing with in general using lograge is trying to reduce my bytes of logfile for papertrail! Drawbacks? if you had some kind of structured log search (I don’t at present, but I guess could with papertrail features by switching to json format?), it might be easier to do something like “find a /search with q=foo and source=ef without worrying about other params) To the extent that params hash can include things not in the actual url, is that important to log like that? ….? Curious what other people think… am I crazy for wanting the actual URL in there, not the params hash? At any rate, it’s pretty easy to do. Note we use filtered_path rather than fullpath to again take account of Rails 6 parameter filtering, and thanks again /u/ezekg: config.lograge.custom_payload do |controller| { path: controller.request.filtered_path } end This is actually overwriting the default path to be one that has the query string too: method=GET path=/search?q=libraries format=html ... You could of course add a different key fullpath instead, if you wanted to keep path as it is, perhaps for easier collation in some kind of log analyzing system that wants to group things by same path invariant of query string. I’m gonna try this out! Meanwhile, on lograge… As long as we’re talking about lograge…. based on commit history, history of Issues and Pull Requests… the fact that CI isn’t currently running (travis.org grr) and doesn’t even try to test on Rails 6.0+ (although lograge seems to work fine)… one might worry that lograge is currently un/under-maintained…. No comment on a GH issue filed in May asking about project status. It still seems to be one of the more popular solutions to trying to tame Rails kind of out of control logs. It’s mentioned for instance in docs from papertrail and honeybadger, and many many other blog posts. What will it’s future be? Looking around for other possibilties, I found semantic_logger (rails_semantic_logger). It’s got similar features. It seems to be much more maintained. It’s got a respectable number of github stars, although not nearly as many as lograge, and it’s not featured in blogs and third-party platform docs nearly as much. It’s also a bit more sophisticated and featureful. For better or worse. For instance mainly I’m thinking of how it tries to improve app performance by moving logging to a background thread. This is neat… and also can lead to a whole new class of bug, mysterious warning, or configuration burden. For now I’m sticking to the more popular lograge, but I wish it had CI up that was testing with Rails 6.1, at least! Incidentally, trying to get Rails to log more compactly like both lograge and rails_semantic_logger do… is somewhat more complicated than you might expect, as demonstrated by the code in both projects that does it! Especially semantic_logger is hundreds of lines of somewhat baroque code split accross several files. A refactor of logging around Rails 5 (I think?) to use ActiveSupport::LogSubscriber made it possible to customize Rails logging like this (although I think both lograge and rails_semantic_logger still do some monkey-patching too!), but in the end didn’t make it all that easy or obvious or future-proof. This may discourage too many other alternatives for the initial primary use case of both lograge and rails_semantic_logger — turn a rails action into one log line, with a structured format. Share this: Twitter Facebook Tagged ruby Published by jrochkind View all posts by jrochkind Published August 4, 2021August 5, 2021 Post navigation Previous Post Notes on Cloudfront in front of Rails Assets on Heroku, with CORS Leave a Reply Cancel reply Enter your comment here... Fill in your details below or click an icon to log in: Email (required) (Address never made public) Name (required) Website You are commenting using your WordPress.com account. ( Log Out /  Change ) You are commenting using your Google account. ( Log Out /  Change ) You are commenting using your Twitter account. ( Log Out /  Change ) You are commenting using your Facebook account. ( Log Out /  Change ) Cancel Connecting to %s Notify me of new comments via email. Notify me of new posts via email. Bibliographic Wilderness is a blog by Jonathan Rochkind about digital library services, ruby, and web development. Contact Search for: Email Subscription Enter your email address to subscribe to this blog and receive notifications of new posts by email. Join 219 other followers Email Address: Subscribe Recent Posts logging URI query params with lograge August 4, 2021 Notes on Cloudfront in front of Rails Assets on Heroku, with CORS June 23, 2021 ActiveSupport::Cache via ActiveRecord (note to self) June 21, 2021 Heroku release phase, rails db:migrate, and command failure June 16, 2021 Code that Lasts: Sustainable And Usable Open Source Code March 23, 2021 Archives Archives Select Month August 2021  (1) June 2021  (3) March 2021  (1) February 2021  (1) January 2021  (4) December 2020  (1) November 2020  (3) October 2020  (2) September 2020  (3) August 2020  (2) April 2020  (1) March 2020  (1) December 2019  (1) October 2019  (1) September 2019  (1) August 2019  (2) June 2019  (2) April 2019  (3) March 2019  (3) February 2019  (1) December 2018  (1) November 2018  (1) October 2018  (2) September 2018  (4) August 2018  (1) June 2018  (2) May 2018  (1) April 2018  (1) March 2018  (3) February 2018  (1) January 2018  (1) November 2017  (1) October 2017  (1) September 2017  (1) August 2017  (3) July 2017  (1) May 2017  (4) April 2017  (2) March 2017  (9) February 2017  (5) January 2017  (1) December 2016  (7) November 2016  (4) September 2016  (1) August 2016  (4) June 2016  (2) May 2016  (4) March 2016  (2) February 2016  (1) January 2016  (2) November 2015  (2) October 2015  (5) September 2015  (7) August 2015  (5) July 2015  (4) May 2015  (3) April 2015  (5) March 2015  (2) February 2015  (2) January 2015  (4) December 2014  (2) November 2014  (2) October 2014  (6) September 2014  (5) August 2014  (3) July 2014  (3) June 2014  (1) May 2014  (3) April 2014  (5) March 2014  (9) February 2014  (4) January 2014  (5) December 2013  (5) November 2013  (14) October 2013  (4) September 2013  (6) August 2013  (2) July 2013  (7) June 2013  (10) May 2013  (4) April 2013  (5) March 2013  (8) February 2013  (6) January 2013  (16) December 2012  (8) November 2012  (14) October 2012  (6) September 2012  (6) August 2012  (2) July 2012  (5) June 2012  (5) May 2012  (7) April 2012  (12) March 2012  (6) February 2012  (7) January 2012  (6) December 2011  (5) November 2011  (7) October 2011  (5) September 2011  (10) August 2011  (4) July 2011  (5) June 2011  (7) May 2011  (8) April 2011  (5) March 2011  (13) February 2011  (4) January 2011  (12) December 2010  (7) November 2010  (5) October 2010  (5) September 2010  (10) August 2010  (6) July 2010  (7) June 2010  (5) May 2010  (8) April 2010  (8) March 2010  (14) February 2010  (3) January 2010  (3) December 2009  (4) November 2009  (2) October 2009  (3) September 2009  (9) August 2009  (1) July 2009  (4) June 2009  (7) May 2009  (14) April 2009  (17) March 2009  (21) February 2009  (11) January 2009  (16) December 2008  (12) November 2008  (30) October 2008  (12) September 2008  (3) July 2008  (4) June 2008  (2) May 2008  (11) April 2008  (3) March 2008  (4) February 2008  (10) January 2008  (7) December 2007  (4) November 2007  (4) September 2007  (1) August 2007  (3) June 2007  (6) May 2007  (12) April 2007  (11) March 2007  (9) Feeds  RSS - Posts  RSS - Comments Recent Comments The Return of the Semantic Web? – Deeply Semantic on “Is the semantic web still a thing?” jrochkind on Rails auto-scaling on Heroku Adam (Rails Autoscale) on Rails auto-scaling on Heroku On catalogers, programmers, and user tasks – Gavia Libraria on Broad categories from class numbers Replacing MARC – Gavia Libraria on Linked Data Caution jrochkind on Deep Dive: Moving ruby projects from Travis to Github Actions for CI jrochkind on Deep Dive: Moving ruby projects from Travis to Github Actions for CI jrochkind on Deep Dive: Moving ruby projects from Travis to Github Actions for CI Top Posts Purposes/Functions of Controlled Vocabulary Bootstrap 3 to 4: Changes in how font size, line-height, and spacing is done. Or "what happened to $line-height-computed." yes, product owner and technical lead need to be different people logging URI query params with lograge Some notes on what's going on in ActiveStorage Top Clicks blog.travis-ci.com/2020-1… aws.amazon.com/premiumsup… kunststube.net/encoding searchstax.com/docs/staxa… github.com/twbs/bootstrap… A blog by Jonathan Rochkind. All original content licensed CC-BY. Create a website or blog at WordPress.com Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use. To find out more, including how to control cookies, see here: Cookie Policy