zpaq on qnap

[fcorbelli]

Does anyone have already compiled zpaq on a QNAP nas?
This seems not a trival task

Using built-in specs.
Target: i686-unknown-linux-gnu
Configured with: ../gcc-4.2.1/configure --build=i386-pc-linux-gnu --host=i686-unknown-linux-gnu --target=i686-unknown-linux-gnu --prefix=/opt --disab
e-nls --disable-static --with-as=/home/slug/optware/ts509/toolchain/i686-unknown-linux-gnu/bin/i686-unknown-linux-gnu-as --with-ld=/home/slug/optware
ts509/toolchain/i686-unknown-linux-gnu/bin/i686-unknown-linux-gnu-ld --enable-languages=c,c++ --disable-multilib
Thread model: posix
gcc version 4.2.1

[Matt Mahoney]

What errors do you get when compiling?

[fcorbelli]

What errors do you get when compiling?

A lot

I'm trying to "split" the Makefile

divsufsort.c compile ok => divsufsort.o

Now...

[/share/CACHEDEV1_DATA/Public/zpaq] # g++ -O3 -Dunix libzpaq.cpp -pthread
/share/CACHEDEV1_DATA/.qpkg/Optware/bin/../lib/gcc/i686-unknown-linux-gnu/4.2.1/../../../../i686-unknown-linux-gnu/lib/crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
/tmp/cc2oUFQX.o: In function `libzpaq::Predictor::assemble_p()':
libzpaq.cpp:(.text+0x575c): undefined reference to `libzpaq::error(char const*)'
libzpaq.cpp:(.text+0x586e): undefined reference to `libzpaq::error(char const*)'
libzpaq.cpp:(.text+0x7eb6): undefined reference to `libzpaq::error(char const*)'
libzpaq.cpp:(.text+0x7ed0): undefined reference to `libzpaq::error(char const*)'
libzpaq.cpp:(.text+0x83dd): undefined reference to `libzpaq::error(char const*)'
/tmp/cc2oUFQX.o:libzpaq.cpp:(.text+0xab88): more undefined references to `libzpaq::error(char const*)' follow
collect2: ld returned 1 exit status

[Bulat Ziganshin]

add "-c" option. all your errors exactly beecause you have splitted compilation

[fcorbelli]

add "-c" option. all your errors exactly beecause you have splitted compilation

"joined" (make) compilation does not run, either, and is very slow.

[/share/CACHEDEV1_DATA/Public/zpaq] # make
gcc -O3 -c -DNDEBUG divsufsort.c
g++ -O3 -Dunix zpaq.cpp libzpaq.cpp divsufsort.o -pthread -o zpaq
/tmp/ccFIi6cM.o: In function `setFilename(char*, int, long long, unsigned int)':
zpaq.cpp:(.text+0x4cb1): undefined reference to `__sync_fetch_and_add_4'
zpaq.cpp:(.text+0x4cc5): undefined reference to `__sync_fetch_and_add_4'
zpaq.cpp:(.text+0x4d88): undefined reference to `__sync_fetch_and_add_4'
zpaq.cpp:(.text+0x4d9c): undefined reference to `__sync_fetch_and_add_4'
/tmp/ccFIi6cM.o: In function `libzpaq::error(char const*)':
zpaq.cpp:(.text+0x4ea5): undefined reference to `__sync_fetch_and_add_4'
/tmp/ccFIi6cM.o:zpaq.cpp:(.text+0x4f07): more undefined references to `__sync_fetch_and_add_4' follow
collect2: ld returned 1 exit status
make: *** [zpaq] Error 1

This seems a problem with platform-specific architecture.
I'll investigate more.

EDIT: after throwing away the Marvel-based NAS , in favor of one based on Intel, and after adding -march=i686 into makefile, I managed to compile zpaq...

[/share/CACHEDEV1_DATA/Public/zpaq] # cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 55
model name      : Intel(R) Celeron(R) CPU  J1800  @ 2.41GHz
stepping        : 8
microcode       : 0x811
cpu MHz         : 2408.000
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx r
dtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16
xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer rdrand lahf_lm 3dnowprefetch ida arat epb dtherm tpr_shadow vnmi flexpriority ept vpid tsc_adj
ust smep erms
bogomips        : 4833.27
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:
(...)

... BUT...

[fcorbelli]

But now the problem is...

[/share/CACHEDEV1_DATA/Public/zpaq] # ./zpaq a ughetto *.cpp
zpaq v6.55 journaling archiver, compiled Oct 28 2014
Appending to ughetto.zpaq at offset 0
Creating ughetto.zpaq version 1 at 2014-10-28 10:02:26
Adding 0.410789 MB in 2 files using 3 jobs in 2 threads.
zpaq error: Config line 2 at hcomp: expected a number
Compression error Config line 2 at hcomp: expected a number

config:
comp 9 16 0 0
hcomp
c-- *c=a a+= 255 d=a *d=c
halt
end


method=x,,5,0,3,
args[0] = $1 = 4
args[1] = $2 = 0
args[2] = $3 = 5
args[3] = $4 = 0
args[4] = $5 = 3
args[5] = $6 = 0
args[6] = $7 = 0
args[7] = $8 = 0
args[8] = $9 = 0
zpaq error: compression error
zpaq exiting from job 1: compression error

Test (of a windows-based zpaq) work fine

[/share/CACHEDEV1_DATA/Public/zpaq] # ./zpaq t ugo.zpaq  -all
zpaq v6.55 journaling archiver, compiled Oct 28 2014
ugo.zpaq: 1 versions, 5 files, 109 fragments, 7.839445 MB
Testing 7.834584 MB in 5 files with 2 jobs
0:00:00 to go: 7.834584 MB (100.00%)
Tested 5 of 5 files OK (0 errors) using 0.068 MB x 2 threads
0.494 seconds (all OK)
[/share/CACHEDEV1_DATA/Public/zpaq] #

[Matt Mahoney]

This is mysterious. It looks like itos() is returning an empty string where it is called from compressBlock(). This results in the malformed "comp 9 16 0 0" line (there should be another 0) and the malformed "method=x,,5,0,3," line (should be something like "method=x4,1,5,0,3,23"). I'm not sure what to make of it. I don't have g++ 4.2.1 but I tried g++ 3.4.5 and it works correctly (in Win32). Is this a 32 bit OS or could it be linking to a 64 bit library?

Edit: try changing line 1171 in itos() in zpaq.cpp to

  for (; x || n>0; x/=10, --n) r=string(1, '0'+x%10)+r;

No idea if this will fix the problem.

[fcorbelli]

This is mysterious. It looks like itos() is returning an empty string where it is called from compressBlock(). This results in the malformed "comp 9 16 0 0" line (there should be another 0) and the malformed "method=x,,5,0,3," line (should be something like "method=x4,1,5,0,3,23"). I'm not sure what to make of it. I don't have g++ 4.2.1 but I tried g++ 3.4.5 and it works correctly (in Win32). Is this a 32 bit OS or could it be linking to a 64 bit library?

Edit: try changing line 1171 in itos() in zpaq.cpp to

  for (; x || n>0; x/=10, --n) r=string(1, '0'+x%10)+r;

No idea if this will fix the problem.

[/share/CACHEDEV1_DATA/Public/zpaq] # make
gcc -O3 -c -DNDEBUG divsufsort.c
g++ -O3 -Dunix zpaq.cpp libzpaq.cpp divsufsort.o -pthread -o zpaq
/tmp/ccbexg60.o: In function `setFilename(char*, int, long long, unsigned int)':
zpaq.cpp:(.text+0x4cb1): undefined reference to `__sync_fetch_and_add_4'
zpaq.cpp:(.text+0x4cc5): undefined reference to `__sync_fetch_and_add_4'
zpaq.cpp:(.text+0x4d88): undefined reference to `__sync_fetch_and_add_4'
zpaq.cpp:(.text+0x4d9c): undefined reference to `__sync_fetch_and_add_4'
/tmp/ccbexg60.o: In function `libzpaq::error(char const*)':
zpaq.cpp:(.text+0x4ea5): undefined reference to `__sync_fetch_and_add_4'
/tmp/ccbexg60.o:zpaq.cpp:(.text+0x4f07): more undefined references to `__sync_fetch_and_add_4' follow
collect2: ld returned 1 exit status
make: *** [zpaq] Error 1

This is a make without any change in Makefile

[fcorbelli]

After a bit of diggin... this seems to work.
Double -march

zpaq: zpaq.cpp libzpaq.cpp libzpaq.h divsufsort.c divsufsort.h
        gcc -march=i686 -O3 -c -DNDEBUG divsufsort.c
        g++ -march=i686 -O3 -Dunix zpaq.cpp libzpaq.cpp divsufsort.o -pthread -o zpaq
        rm divsufsort.o