Skip to content

Build failure on raspberry pi running arch #105

Closed

Description

@mike-lawrence

Log:

    [user@scene_pi python-blosc]$ sudo python setup.py install
    running install
    running bdist_egg
    running egg_info
    writing dependency_links to blosc.egg-info/dependency_links.txt
    writing top-level names to blosc.egg-info/top_level.txt
    writing blosc.egg-info/PKG-INFO
    reading manifest file 'blosc.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.txt'
    warning: no files found matching '*.cpp' under directory 'c-blosc'
    warning: no files found matching '*.hpp' under directory 'c-blosc'
    writing manifest file 'blosc.egg-info/SOURCES.txt'
    installing library code to build/bdist.linux-armv7l/egg
    running install_lib
    running build_py
    creating build
    creating build/lib.linux-armv7l-3.4
    creating build/lib.linux-armv7l-3.4/blosc
    copying blosc/test.py -> build/lib.linux-armv7l-3.4/blosc
    copying blosc/__init__.py -> build/lib.linux-armv7l-3.4/blosc
    copying blosc/version.py -> build/lib.linux-armv7l-3.4/blosc
    copying blosc/toplevel.py -> build/lib.linux-armv7l-3.4/blosc
    running build_ext
    building 'blosc.blosc_extension' extension
    creating build/temp.linux-armv7l-3.4
    creating build/temp.linux-armv7l-3.4/blosc
    creating build/temp.linux-armv7l-3.4/c-blosc
    creating build/temp.linux-armv7l-3.4/c-blosc/blosc
    creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs
    creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs/lz4-1.7.0
    creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs/snappy-1.1.1
    creating build/temp.linux-armv7l-3.4/c-blosc/internal-complibs/zlib-1.2.8
    gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-1.7.0 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/usr/include/python3.4m -c blosc/blosc_extension.c -o build/temp.linux-armv7l-3.4/blosc/blosc_extension.o
    gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-1.7.0 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/usr/include/python3.4m -c c-blosc/blosc/blosc.c -o build/temp.linux-armv7l-3.4/c-blosc/blosc/blosc.o
    c-blosc/blosc/blosc.c: In function 'blosc_getitem':
    c-blosc/blosc/blosc.c:1275:7: warning: unused variable 'tmp_init' [-Wunused-variable]
       int tmp_init = 0;
           ^
    gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-1.7.0 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/usr/include/python3.4m -c c-blosc/blosc/shuffle-sse2.c -o build/temp.linux-armv7l-3.4/c-blosc/blosc/shuffle-sse2.o
    c-blosc/blosc/shuffle-sse2.c:14:4: error: #error SSE2 is not supported by the target architecture/platform and/or this compiler.
       #error SSE2 is not supported by the target architecture/platform and/or this compiler.
        ^
    c-blosc/blosc/shuffle-sse2.c:17:23: fatal error: emmintrin.h: No such file or directory
    compilation terminated.
    error: command 'gcc' failed with exit status 1

Activity

  1. FrancescAlted commented on Sep 23, 2015

    @FrancescAlted
    Member

    Hi,

    Yes, python-blosc is being developed in Intel and normally only checked with it. To overcome this problem we need to recognize the architecture prior to call the C compiler: this would allow to set compiler options accordingly. PR welcome!

  2. mike-lawrence commented on Sep 24, 2015

    @mike-lawrence
    Author

    Ah, I thought that because c-blosc is supposedly compatible with arm/RPi (at least, according to the cblosc 1.3.6 to 1.4.0 change notes), that python-blosc would be as well.

  3. esc commented on Oct 24, 2016

    @esc
    Member

    Any news on this? Perhaps a new version has fixed the issue? Or maybe it magically disappeared?

  4. itdaniher commented on Jan 30, 2017

    @itdaniher
    Contributor

    I had a surprising amount of luck with blosc on a recent ARM core. Eagerly looking forward to c-blosc2's NEON support, but figured I'd chime in and suggest that there no fundamental issues with c-blosc or python-blosc on ARM.

    Some stats and comparisons below.

    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    python-blosc version: 1.4.4
    Blosc version: 1.11.2 ($Date:: 2017-01-27 #$)
    Compressors available: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']
    Compressor library versions:
      BloscLZ: 1.0.5
      LZ4: 1.7.5
      Snappy: 1.1.1
      Zlib: 1.2.8
      Zstd: 1.1.2
    Python version: 3.6.0 (default, Dec 31 2016, 21:20:16)
    [GCC 4.9.2]
    Platform: Linux-3.4.113-sun8i-armv7l (#50 SMP PREEMPT Mon Nov 14 08:41:55 CET 2016)
    Linux dist: debian 9.0
    Processor: not recognized
    Byte-ordering: little
    Detected cores: 4
    Number of threads to use by default: 4
    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
      *** ctypes.memmove() *** Time for memcpy():   0.015 s (93.57 MB/s)
    
    Times for compressing/decompressing with clevel=5 and 4 threads
    
    *** user input ***
      *** blosclz , noshuffle  ***  0.015 s (89.93 MB/s) / 0.010 s (138.32 MB/s)    Compr. ratio:   2.7x
      *** blosclz , shuffle    ***  0.023 s (60.25 MB/s) / 0.012 s (112.71 MB/s)    Compr. ratio:   2.3x
      *** blosclz , bitshuffle ***  0.018 s (77.63 MB/s) / 0.021 s (66.76 MB/s)     Compr. ratio:   7.3x
      *** lz4     , noshuffle  ***  0.008 s (177.14 MB/s) / 0.009 s (159.00 MB/s)   Compr. ratio:   3.6x
      *** lz4     , shuffle    ***  0.010 s (131.29 MB/s) / 0.012 s (117.69 MB/s)   Compr. ratio:   3.5x
      *** lz4     , bitshuffle ***  0.015 s (89.97 MB/s) / 0.022 s (63.62 MB/s)     Compr. ratio:   8.4x
      *** lz4hc   , noshuffle  ***  0.071 s (19.30 MB/s) / 0.007 s (186.64 MB/s)    Compr. ratio:   8.6x
      *** lz4hc   , shuffle    ***  0.079 s (17.30 MB/s) / 0.014 s (95.99 MB/s)     Compr. ratio:   6.2x
      *** lz4hc   , bitshuffle ***  0.062 s (22.23 MB/s) / 0.027 s (51.53 MB/s)     Compr. ratio:   9.7x
      *** snappy  , noshuffle  ***  0.008 s (173.87 MB/s) / 0.009 s (148.77 MB/s)   Compr. ratio:   4.4x
      *** snappy  , shuffle    ***  0.011 s (123.22 MB/s) / 0.016 s (85.16 MB/s)    Compr. ratio:   4.4x
      *** snappy  , bitshuffle ***  0.015 s (89.02 MB/s) / 0.021 s (64.87 MB/s)     Compr. ratio:   6.2x
      *** zlib    , noshuffle  ***  0.047 s (29.26 MB/s) / 0.011 s (121.83 MB/s)    Compr. ratio:  14.7x
      *** zlib    , shuffle    ***  0.080 s (17.20 MB/s) / 0.022 s (63.61 MB/s)     Compr. ratio:   9.4x
      *** zlib    , bitshuffle ***  0.059 s (23.50 MB/s) / 0.033 s (41.10 MB/s)     Compr. ratio:  10.5x
      *** zstd    , noshuffle  ***  0.113 s (12.21 MB/s) / 0.011 s (124.64 MB/s)    Compr. ratio:  15.6x
      *** zstd    , shuffle    ***  0.154 s (8.92 MB/s) / 0.026 s (52.56 MB/s)      Compr. ratio:   9.9x
      *** zstd    , bitshuffle ***  0.116 s (11.86 MB/s) / 0.036 s (38.40 MB/s)     Compr. ratio:  11.4x
    

    Script is a slightly modified version of compress_ptr.py -
    https://gist.github.com/itdaniher/544215e2ebd25495a95d9165633972c6

    Tried a few different formats for my sampled analog signal compression - pysoundfile's ogg and flac encodings, python-lz4, and python's integrated zlib. Very happy with Blosc overall - stellar performance even without native SIMD (yet!)

  5. itdaniher commented on Jan 30, 2017

    @itdaniher
    Contributor

    @FrancescAlted I'd be glad to order your choice of single board computer to better enable you to test on ARM CPUs. Also have plenty of recommendations, the ability to offer native hosted ARM continuous integration, etc. Contact me @gmail.com if you're interested! Thanks for Blosc-and-friends!

  6. FrancescAlted commented on Mar 10, 2017

    @FrancescAlted
    Member

    @itdaniher Thanks for reporting. As you had no fundamental flaws with C-Blosc in ARM, I am going to close this one. I already contacted you about the ARM board and CI for ARM. Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions