create_llvm_prof is not working when perf version is 6.12

Hi together,

Recently on cachyos we have updated the linux-tools perf to the 6.12 release.
When trying to convert the profile with perf 6.12 I get following:

 create_llvm_prof --binary=/usr/src/debug/linux-cachyos/vmlinux --profile=/home/ptr1337/bench/withoutprime.data --format=extbinary --out=cachyos-benchmarker.afdo
[ERROR:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:161] Event size 0 of event UNKNOWN_EVENT_0 is less than header size 8
[ERROR:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1335] Error reading build ID header.
WARNING: Logging before InitGoogleLogging() is written to STDERR
E20241205 17:53:35.636153 254754 profile_creator.cc:209] Error reading profile from /home/ptr1337/bench/withoutprime.data

When using perf 6.10 it does work correctly:

create_llvm_prof --binary=/usr/src/debug/linux-cachyos/vmlinux --profile=/home/ptr1337/bench/withoutprime.data --format=extbinary --out=cachyos-benchmarker.afdo
[WARNING:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1322] Skipping 268 bytes of metadata: HEADER_CPU_TOPOLOGY
[WARNING:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1069] Skipping unsupported event PERF_RECORD_ID_INDEX
[WARNING:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1069] Skipping unsupported event PERF_RECORD_EVENT_UPDATE
[WARNING:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1069] Skipping unsupported event PERF_RECORD_CPU_MAP
[WARNING:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1069] Skipping unsupported event UNKNOWN_EVENT_17
[WARNING:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1069] Skipping unsupported event UNKNOWN_EVENT_82
[INFO:/home/runner/work/autofdo/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1060] Number of events stored: 4931669

Reproducer:

Go into cachyos or archlinux enviroment
Compile kernel with autofdo support and an unstripped vmlinux
Boot into the kernel and profile a workload with perf record --pfm-events RETIRED_TAKEN_BRANCH_INSTRUCTIONS:k -a -N -b -c 500009 -o cachyos-benchmarker.data -- cachyos-benchmarker ~/bench
Try to convert the profile: create_llvm_prof --binary=/usr/src/debug/linux-cachyos/vmlinux --profile=/home/ptr1337/bench/withoutprime.data --format=extbinary --out=cachyos-benchmarker.afdo

If you need further logs, please let me know

Little update:
perf 6.11 is working. So its a regression/change in the 6.12 release.

If you could help me debugging this down, I would like to report ustream

Thanks for reporting. We are looking into it and will report back shortly.

Hi Peter, I am trying to reproduce.

I worked on a archlinux, updated to newest with kernel 6.12.4 and perf 6.12-1. Seems the system is missing libpfm support, and I cannot find libpfm via pacman. Then I saw you patch to add support and I tried to makepkg on your change but failed with libpfm not found error on my archlinux. Could you briefly describe how to add libpfm support for arch, thanks.

-Han

Hi Peter, I am trying to reproduce.

I worked on a archlinux, updated to newest with kernel 6.12.4 and perf 6.12-1. Seems the system is missing libpfm support, and I cannot find libpfm via pacman. Then I saw you patch to add support and I tried to makepkg on your change but failed with libpfm not found error on my archlinux. Could you briefly describe how to add libpfm support for arch, thanks.

-Han

Hi,

I have pushed a perf version with working pfm support now into the extra-testing repository.
Please fetch the libpfm and perf package from there. I have verified that pfm events are working.

You can find them here (Click the download button and then install with sudo pacman -U):
https://archlinux.org/packages/extra-testing/x86_64/libpfm/
https://archlinux.org/packages/extra-testing/x86_64/perf/

Hi Peter, thanks for preparing the packages. I am now able to reproduce.

The perf data parser, that this tool depends on, failed to parse "struct perf_event_header" (all fields are filled with zero), this may be caused by (unlikely) the new perf-6.12 tool's failing to correctly write the header into perf.data, or (probably) the parser has a bug or is incapable of parsing the new perf-6.12. format (I don't know what has changed in 6.12). I'll keep looking.

Filed an internal bug regarding perf data parser, will report back.

We found the error happens during reading build-id, and we then found this.

After apply the patch on top of linux-tools-6.12-7 , the "Error reading build ID header." is gone.

diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c                                                                                          
index 8982f68e7230..4baf925beb1f 100644                                                                                                                       
--- a/tools/perf/util/build-id.c                                                                                                                              
+++ b/tools/perf/util/build-id.c                                                                                                                              
@@ -277,7 +277,8 @@ static int write_buildid(const char *name, size_t name_len, struct build_id *bid                                                          
        struct perf_record_header_build_id b;                                                                                                                 
        size_t len;                                                                                                                                           
                                                                                                                                                              
-       len = sizeof(b) + name_len + 1;                                                                                                                       
+       /* len = sizeof(b) + name_len + 1; */                                                                                                                 
+       len = name_len + 1;                                                                                                                                   
        len = PERF_ALIGN(len, sizeof(u64));                                                                                                                   
                                                                                                                                                              
        memset(&b, 0, sizeof(b));                                                                                                                             
@@ -286,7 +287,8 @@ static int write_buildid(const char *name, size_t name_len, struct build_id *bid                                                          
        misc |= PERF_RECORD_MISC_BUILD_ID_SIZE;                                                                                                               
        b.pid = pid;                                                                                                                                          
        b.header.misc = misc;                                                                                                                                 
-       b.header.size = len;                                                                                                                                  
+       /* b.header.size = len; */                                                                                                                            
+       b.header.size = sizeof(b) + len;                                                                                                                      
                                                                                                                                                              
        err = do_write(fd, &b, sizeof(b));                                                                                                                    
        if (err < 0)

@captain5050

cc @captain5050, perhaps a bug in https://github.com/googleprodkernel/linux-perf/commit/ae39ba16554eb4a2d97f752847e93aa428438912.

I believe this is fixed by: https://lore.kernel.org/r/Z0aRFFW9xMh3mqKB@google.com

Does that need to get cherry-picked back into 6.12?

Thanks! The fix appears to work from Namhyung Kim

The fixes tag means it should be pulled automatically.

upgpkg: 6.12-9, add fix for build id

Navigation Menu

Repository navigation

create_llvm_prof is not working when perf version is 6.12 #233

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

create_llvm_prof is not working when perf version is 6.12 #233

Description

Activity

ptr1337 commented on Dec 11, 2024

shenhanc78 commented on Dec 12, 2024

shenhanc78 commented on Dec 12, 2024

ptr1337 commented on Dec 13, 2024

shenhanc78 commented on Dec 13, 2024

shenhanc78 commented on Dec 14, 2024

shenhanc78 commented on Dec 17, 2024

nickdesaulniers commented on Dec 18, 2024

captain5050 commented on Dec 18, 2024

nickdesaulniers commented on Dec 19, 2024

ptr1337 commented on Dec 19, 2024

captain5050 commented on Dec 19, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions