bug#29654: GDBM output is not deterministic

[Ludovic Courtès]

Hello Sergey & co.,

While investigating <https://bugs.gnu.org/29654>, we realized that
databases produced by GDBM are not bit-for-bit reproducible.  This can
be seen with this program:

--8<---------------cut here---------------start------------->8---
#include <unistd.h>
#include <gdbm.h>

int
main ()
{
  unlink ("/tmp/t.db");
  gdbm_open ("/tmp/t.db", 0, GDBM_WRCREAT, 0644, NULL);
}
--8<---------------cut here---------------end--------------->8---

Under Valgrind, we get:

--8<---------------cut here---------------start------------->8---
==15981== Syscall param write(buf) points to uninitialised byte(s)
==15981==    at 0x5335080: __write_nocancel (in 
/gnu/store/3h31zsqxjjg52da5gp3qmhkh4x8klhah-glibc-2.25/lib/libc-2.25.so)
==15981==    by 0x4E3E44D: _gdbm_full_write (in 
/gnu/store/kg8ffb14msfnc9aivxj6djrl51g9b3zz-gdbm-1.13/lib/libgdbm.so.4.0.0)
==15981==    by 0x4E3B6AD: gdbm_fd_open (in 
/gnu/store/kg8ffb14msfnc9aivxj6djrl51g9b3zz-gdbm-1.13/lib/libgdbm.so.4.0.0)
==15981==    by 0x400743: main (in /home/ludo/src/guix/a.out)
==15981==  Address 0x55fb204 is 4 bytes inside a block of size 4,096 alloc'd
==15981==    at 0x4C2AAD6: malloc (in 
/gnu/store/p2b1rzqlpdqbhn42g76xzgykbivwc063-valgrind-3.12.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==15981==    by 0x4E3B5E6: gdbm_fd_open (in 
/gnu/store/kg8ffb14msfnc9aivxj6djrl51g9b3zz-gdbm-1.13/lib/libgdbm.so.4.0.0)
==15981==    by 0x400743: main (in /home/ludo/src/guix/a.out)
--8<---------------cut here---------------end--------------->8---

… which suggests that an uninitialized block is being written straightly
to disk.

If we tell libc to initialize ‘malloc’ areas that were otherwise left
uninitialized, we can confirm the problem (again with the program
above):

--8<---------------cut here---------------start------------->8---
$ for i in `seq 0 20` ; do MALLOC_PERTURB_=$i ./a.out ; sha256sum /tmp/t.db ; 
done
0483a04a43d0a08a5a0aa71d09b262b62eb2504e85c5fe7104b783a5f0f7ae15  /tmp/t.db
29122b2994face5b5a716c8cd0e6beda7f05c3441441447872c2f15e7e2db308  /tmp/t.db
34a130820ab6d5e638adae55e83cb784fec036c2da4727a113d3d1191331bd7c  /tmp/t.db
9604ce76ea1f1b37968bd037798c0eb212a1e654f060b7c3d66011620b249212  /tmp/t.db
a1c275c1bf1b5980166431bd7245d2d649499a8cad233059abf2a6cbf2dec19b  /tmp/t.db
1cbe43f146a158ac4c21064feac1371e7a6bb4ac9d0013b15acfe7fec754486f  /tmp/t.db
83778759475790f1377adadd77b9d2eee7db7e5e43e006ef04e805f4b4309590  /tmp/t.db
6b80d00119fd806d05f8863056919ea0a6b66a0771d16439b072e2cc2eac6b58  /tmp/t.db
f1f65b95e39f74cff12c3e830eea77a9841c8793f4441c6a8bfa9ad92d765d8d  /tmp/t.db
3dd2479f44c0be15e02f0040fcef3d765a306dbd2a47b3557cb19fb1282b0b05  /tmp/t.db
fed4cc15d84f676737954e80ab594034e47d2f2b7cbcd12dad722dddff6c5afb  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
3d9bccf127fd76b56617435c412aeec0c8fc95ab6b9d5f9e3f24060398899c26  /tmp/t.db
743808bd91a42e2c149a30bbdfacadbaa4124f82e01ea43323d5c1fd10de5499  /tmp/t.db
86a92606713ca03cab09f1cc97463184ed0c4d2051d7512dd98b136bc7ca1465  /tmp/t.db
93f5428a4fde073cab1737c1efd64e2554731db1ab51d77aec95c81592b014a0  /tmp/t.db
b93fd6692eeebfc5b49b64857de7647a88e541be7d4fd4e62fb6771cb4b00618  /tmp/t.db
ba08b6acee5812c3b3770f842aca7435729acc1082cddf8bfe26024f9ea8f54e  /tmp/t.db
30dc2e3d1ea0150e4a200435b1f609de726680ebfc8661a9d121afbd4cf0b0a8  /tmp/t.db
0de2cc18e445ff0a2697cd2783f379fa202d57020db9a73c55c4f7b8fd5cac74  /tmp/t.db
dd938784d3799c5a575abc3fe24191f3e1ac593643719a9694f9fed32ed272da  /tmp/t.db
$ for i in `seq 0 20` ; do MALLOC_PERTURB_=11 ./a.out ; sha256sum /tmp/t.db ; 
done
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc  /tmp/t.db
--8<---------------cut here---------------end--------------->8---

I guess the fix is to use ‘calloc’ instead of ‘malloc’ for this specific
allocation.

WDYT?

Ludo’.