Skip to content

Enhance the access log format #69

Closed
Closed
Enhance the access log format#69
Assignees
etr
Labels
feature-requestFeature requests or enhancements

Description

@craigminihan
craigminihan
opened on Feb 24, 2015

Currently the access log outputs the following format:

/_utils/index.html METHOD: GET
/_utils/style/layout.css?0.11.0 METHOD: GET
/_utils/script/jquery.js METHOD: GET
/_utils/script/jquery.dialog.js METHOD: GET
/_utils/script/json2.js METHOD: GET

It would be useful if a standard format was used, the most obvious being: http://en.wikipedia.org/wiki/Common_Log_Format. A more adventurous alternate would be the MS IIS extended log file format: https://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/bea506fd-38bc-4850-a4fb-e3a0379d321f.mspx?mfr=true, this is a customizable format which has the interesting time-taken field which is critical for performance analysis of REST APIs.

Activity

  1. etr commented on Feb 25, 2015

    @etr
    etr
    on Feb 25, 2015
    Owner

    I think that using a standard logging format is more than just needed when building a web server.

    The library itself though is not a web server, but a tool to build one. The responsibility of the library (for how it is designed) toward logging it's to provide a support to plug a logging system.

    Right now, when constructing a webserver object, it is possible to pass handlers for access_log and error_log.

    The log lines you are attaching to your comment come from a mis-implementation of the logging in the library (it is a refuse from the early stages of development when everything was a single big method).

    I think the right thing to do, would be to improve the interface of access_log and error_log so that (at least) a standard log format could be used.

    Said so, I do not think that building a logger is totally out of board - but I think it should be isolated from the rest of the library and plugged using the interface (maybe shipped with the library as a default replaceable logger).

  2. craigminihan commented on Feb 25, 2015

    @craigminihan
    craigminihan
    on Feb 25, 2015
    ContributorAuthor

    @etr I've looked at webserver::answer_to_connection where access_log is called from. It looks to me that other than url/method/version/etc passed as params to answer there isn't any other useful information available. It would be nice to get MHD_Connection* connection but this is an internal decl in libmicrohttpd which is a shame.

  3. etr
    self-assigned this
    on Jan 15, 2019
  4. etr
    added
    feature-requestFeature requests or enhancements
    on Jan 15, 2019

  5. fechantillac-antidot commented on Jul 8, 2020

    @fechantillac-antidot
    fechantillac-antidot
    on Jul 8, 2020
    Contributor

    Interested in this too. Migrating from fastcgi to libhttpserver, given the lack of information in the current access log we plan to keep using apache as a proxy. What would be useful would be to get some informations about the response in the access log callback. At least the status code. Size and response time would be quite useful as well.

    Is it something you may be addressing in the current year ?

    FYI we are using spdlog as a logging facility.

  6. etr commented on Jul 8, 2020

    @etr
    etr
    on Jul 8, 2020
    Owner

    One of the main feature I want to release this year is a system of interceptors at global ws level (pre-routing, post-routing) and at resource level (pre-handling, post-handling). The initial intent was to tackle cases like ( #102 ). That should allow for any level of elasticity because those interceptor would get the full http_request in input (so from there you could log anything really).

    I think I will keep the access_log method around, but that is likely going to be limited because it is called before the request is even parsed (despite having the benefit that it is very hard for it to ever miss anything). I might enrich that with a better set of data, but as said, it would be limited there.

    For your case, I think that the interceptors should help the most.

  7. EverydayDiesel commented on Sep 29, 2022

    @EverydayDiesel
    EverydayDiesel
    on Sep 29, 2022

    I would also like to see this functionality. I am still trying to figure out how to get the ip address of the requester

  8. etr commented on Sep 29, 2022

    @etr
    etr
    on Sep 29, 2022
    Owner

    @EverydayDiesel you should already be able to access the ip address of the requester using the get_requestor method from the http_request; see: http://github.com/etr/libhttpserver#parsing-requests

  9. EverydayDiesel commented on Oct 9, 2022

    @EverydayDiesel
    EverydayDiesel
    on Oct 9, 2022

    @EverydayDiesel you should already be able to access the ip address of the requester using the get_requestor method from the http_request; see: http://github.com/etr/libhttpserver#parsing-requests

    Hello so sorry to bother you. Is it possible to get the user agent ie something like this?
    Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

    I tried all of these but never got it

        std::cout << "IP Address: " << req.get_requestor() << std::endl;
    std::cout << "Get Version: " << req.get_version() << std::endl;
    std::cout << "Requestor: " << req.get_requestor() << std::endl;
    std::cout << "Requestor Port: " << req.get_requestor_port() << std::endl;
    std::cout << "Version: " << req.get_version() << std::endl;
    std::cout << "Content: " << req.get_content() << std::endl;

  10. etr commented on Oct 10, 2022

    @etr
    etr
    on Oct 10, 2022
    Owner

    This is getting off topic from what this thread was about, so if you need further info on how to use the APIs, please open a separate one or use gitter.

    The user agent is a header, not a detail of the connection. As such, you can access it through the headers within the request, provided that your client is passing them. The get_header and get_headers methods in request (http://github.com/etr/libhttpserver#parsing-requests) are what you are looking for.

  11. Lotharyx commented on Mar 14, 2023

    @Lotharyx
    Lotharyx
    on Mar 14, 2023

    One of the main feature I want to release this year [2020] is a system of interceptors at global ws level (pre-routing, post-routing) and at resource level (pre-handling, post-handling).

    Did this get released in the past couple years? I'm (seemingly) stuck having all of my resource classes call a common logging method (which takes the request object as an argument) to get the log info formatted the way I want. Is there a better way?

  12. etr
    added 3 commits that reference this issue on May 21, 2026

    specs: lifecycle hook bus (DR-012 + TASK-045..052)

    6eb3c4c

    TASK-050: fire after_handler / response_sent / request_completed + lo…

    bed6ad3

    TASK-052: Hook bus doc/example/bench/stress closeout

    830a929

  13. etr commented on Jul 23, 2026

    @etr
    etr
    on Jul 23, 2026
    Owner

    Resolved in libhttpserver 2.0.0. The new lifecycle-hook system adds a response_sent hook that fires after each response with the HTTP status code, bytes queued, and elapsed time — enough to emit Common Log Format including the time-taken field you mentioned. There's a ready-made example at examples/clf_access_log.cpp, and the hooks are documented in the README's Lifecycle Hooks section.

    Closing as resolved by 2.0.0.

  14. etr
    closed this as completedon Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

feature-requestFeature requests or enhancements

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions