D-Bus main loop programming

Over internet, there are many D-Bus examples like the following

1
2
3
4
5
6
7
8
9
10
11
12
13

while(true)
{
    dbus_connection_read_write();
    …
    pMessage = dbus_connection_pop_message();

    if (pMessage)
    {
        …
        dbus_message_unref(pMessage);
    }

}

This is incorrect.

• dbus_connection_read_write() is a low level function bringing in raw data from lower transport layer.  It does not parse data and not aware of how many messages have been brought up.
• dbus_message_pop_message() is a function popping 1 message from incoming queue/buffer. 

Overall, each dbus_connection_read_write() can effectively bring in more than 1 message.  Consequently, only popping 1 message after each dbus_connection_read_write() will potentially leave a large number of data in the incoming queue/buffer.

The correct loop is following

1
2
3
4
5
6
7
8
9
10
11
12
13

while(true)
{
    dbus_connection_read_write();
    …
    pMessage = dbus_connection_pop_message();

    while (pMessage)
    {
        …
        dbus_message_unref(pMessage);
    }

}

Build Ubuntu Package from Source

TBC...

root# apt-get source dbus
Reading package lists... Done
Building dependency tree    
Reading state information... Done
NOTICE: 'dbus' packaging is maintained in the 'Git' version control system at:
git://anonscm.debian.org/pkg-utopia/dbus.git
Need to get 1,961 kB of source archives.
Get:1 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main dbus 1.4.18-1ubuntu1.7 (dsc) [2,614 B]
Get:2 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main dbus 1.4.18-1ubuntu1.7 (tar) [1,893 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main dbus 1.4.18-1ubuntu1.7 (diff) [64.8 kB]
Fetched 1,961 kB in 1s (1,304 kB/s)
gpgv: Signature made Tue 25 Nov 2014 03:24:17 PM EST using RSA key ID A744BE93
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./dbus_1.4.18-1ubuntu1.7.dsc
dpkg-source: info: extracting dbus in dbus-1.4.18
dpkg-source: info: unpacking dbus_1.4.18.orig.tar.gz
dpkg-source: info: unpacking dbus_1.4.18-1ubuntu1.7.debian.tar.gz
dpkg-source: info: applying 01_no-fatal-warnings.patch
dpkg-source: info: applying 02_obsolete_g_thread_api.patch
dpkg-source: info: applying 20_system_conf_limit.patch
dpkg-source: info: applying 81-session.conf-timeout.patch
dpkg-source: info: applying 0001-activation-allow-for-more-variation-than-just-system.patch
dpkg-source: info: applying 0002-bus-change-systemd-activation-to-activation-systemd.patch
dpkg-source: info: applying 0003-upstart-add-upstart-as-a-possible-activation-type.patch
dpkg-source: info: applying 0004-upstart-add-UpstartJob-to-service-desktop-files.patch
dpkg-source: info: applying 0005-activation-implement-upstart-activation.patch
dpkg-source: info: applying CVE-2012-3524-dbus.patch
dpkg-source: info: applying CVE-2012-3524-regression-fix.patch
dpkg-source: info: applying CVE-2013-2168.patch
dpkg-source: info: applying CVE-2014-3477.patch
dpkg-source: info: applying CVE-2014-3532.patch
dpkg-source: info: applying CVE-2014-3533.patch
dpkg-source: info: applying CVE-2014-3635.patch
dpkg-source: info: applying CVE-2014-3636.patch
dpkg-source: info: applying CVE-2014-3637.patch
dpkg-source: info: applying CVE-2014-3638.patch
dpkg-source: info: applying CVE-2014-3639.patch
dpkg-source: info: applying CVE-2014-7824.patch
dpkg-source: info: applying CVE-2014-3639-regression.patch

# cd dbus-1.4.18
# ./configure --prefix=
# make

Files:
mkdir -p ~/dbus/bin
cp bus/dbus-daemon ~/dbus/bin
cp tools/dbus-cleanup-sockets ~/dbus/bin
cp tools/dbus-uuidgen ~/dbus/bin

mkdir -p ~/dbus/usr/bin
cp tools/dbus-monitor ~/dbus/usr/bin
cp tools/dbus-send ~/dbus/usr/bin

mkdir -p ~/dbus/lib/i386-linux-gnu
cp dbus/.libs/libdbus-1.so.3.5.8 ~/dbus/lib/i386-linux-gnu

mkdir -p ~/dbus/usr/lib/dbus-1.0
cp bus/dbus-daemon-launch-helper ~/dbus/lib/dbus-1.0/

Mac OS X can not sign in App Store / iTunes / iCloud

Validated on Mac OS X El Capitan


Run the following commands:
sudo pkill -9 -f Account
sudo rm $HOME/Library/Accounts/*
sudo pkill -9 -f Store
sudo pkill -9 -f iTunes

Restart App Store and log on

D-Bus memory leak

My goodness.  This whole week, the whole team have been scrambling around memory leak related to D-Bus.

Some basic references:
http://dbus.freedesktop.org/doc/dbus-tutorial.html
http://sy198704.is-programmer.com/posts/33060.html
http://www.cnblogs.com/chenbin7/archive/2013/03/05/2944895.html

We have a Ubuntu 12.04 LTS.  The D-Bus version is 1.4.18.  There are a good number of leak issues.  What should we do?

Well, desirably, we should go to 1.7.10.


Here is the list of leak issues.

might leak memory in dbus_message_iter_get_args_valist:
report date: 4/17/2009
fixed 1.7.8
https://bugs.freedesktop.org/show_bug.cgi?id=21259


file dicriptor leak in _dbus_connect_tcp_socket_with_nonce
report on 5/16/2011
fixed in 1.5.6
https://bugzilla.freedesktop.org/show_bug.cgi?id=37258


a small memory leak, and a failure to report errors, when updating a service file entry for activation
report on 7/21/2011
fixed in 1.5.8
https://bugzilla.freedesktop.org/show_bug.cgi?id=39230


can crash on failed realloc; cannot be forced to crash on all failed mallocs
report on 9/20/2011
fixed in 1.5.10
https://bugzilla.freedesktop.org/show_bug.cgi?id=41048


Dbus is leaking some fd to /dev/null when it demonize.
report on 11/9/2012
fixed in 1.7.0
https://bugzilla.freedesktop.org/show_bug.cgi?id=56927


File descriptor leak in _dbus_command_for_pid():
report on 9/10/2013
fixed in 1.6.16, 1.7.6
https://bugs.freedesktop.org/show_bug.cgi?id=69182


Memory leak in dbus-daemon:
report on 11/12/2013
fixed in 1.6.20, 1.7.10
https://bugs.freedesktop.org/show_bug.cgi?id=71526


D-Bus 1.8.0 release note (1/20/2014)
• fixed long-standing fd and array leaks when failing to parse a message
• fixed referenced-but-never-freed parent nodes (effectively memory
  leaks) when using certain object-path allocation patterns, notably in
  Avahi

dbus-monitor leak file descriptor
report on 6/27/2014
fixed in 1.9.0
https://bugs.freedesktop.org/show_bug.cgi?id=80603


Missing varargs cleanup
report on 1/6/2015
fixed in 1.9.8
https://bugs.freedesktop.org/show_bug.cgi?id=88087


Memleak when GetConnectionCredentials is successful
report on 6/17/2015
fixed in 1.8.20, 1.9.18
https://bugs.freedesktop.org/show_bug.cgi?id=91008


libcap-ng < 0.7.7 leaks one non-close-on-exec fd during initialization
report on 8/19/2015
fixed in 1.10
https://bugs.freedesktop.org/show_bug.cgi?id=91684

recover MBR

For dual boot installation.

http://www.sysresccd.org/SystemRescueCd_Homepage

dd if=/dev/sda of=mbr.bin bs=512 count=1

http://www.digitalcitizen.life/create-usb-memory-stick-system-recovery-tools

http://www.sevenforums.com/tutorials/20864-mbr-restore-windows-7-master-boot-record.html

grub.cfg windows entry

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'Windows 8 (loader) (on /dev/sda1)' --class windows --class os $menuentry_id_option 'osprober-chain-A8A8863CA88608D0' {
 insmod part_msdos
 insmod ntfs
 set root='hd0,msdos1'
 if [ x$feature_platform_search_hint = xy ]; then
   search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  A8A8863CA88608D0
 else
   search --no-floppy --fs-uuid --set=root A8A8863CA88608D0
 fi
 parttool ${root} hidden-
 drivemap -s (hd0) ${root}
 chainloader +1
}
menuentry 'Windows Recovery Environment (loader) (on /dev/sdb3)' --class windows --class os $menuentry_id_option 'osprober-chain-1ACA-FBDA' {
 insmod part_gpt
 insmod fat
 set root='hd1,gpt3'
 if [ x$feature_platform_search_hint = xy ]; then
   search --no-floppy --fs-uuid --set=root --hint-bios=hd1,gpt3 --hint-efi=hd1,gpt3 --hint-baremetal=ahci1,gpt3  1ACA-FBDA
 else
   search --no-floppy --fs-uuid --set=root 1ACA-FBDA
 fi
 drivemap -s (hd0) ${root}
 chainloader +1
}
menuentry 'Windows 8 (loader) (on /dev/sdb5)' --class windows --class os $menuentry_id_option 'osprober-chain-6A18CDFA18CDC573' {
 insmod part_gpt
 insmod ntfs
 set root='hd1,gpt5'
 if [ x$feature_platform_search_hint = xy ]; then
   search --no-floppy --fs-uuid --set=root --hint-bios=hd1,gpt5 --hint-efi=hd1,gpt5 --hint-baremetal=ahci1,gpt5  6A18CDFA18CDC573
 else
   search --no-floppy --fs-uuid --set=root 6A18CDFA18CDC573
 fi
 drivemap -s (hd0) ${root}
 chainloader +1
}

Linux Memory Management

32 bit : http://tldp.org/HOWTO/KernelAnalysis-HOWTO-7.html
64 bit : https://www.kernel.org/doc/gorman/html/understand/understand012.html

Check Memory Utilization
cat /proc/meminfo
free -m

Configuration
/proc/sys/vm/*

Dirty Page
/proc/sys/vm/dirty_background_ratio: disk write buffer flushing threshold as percentage of system memory
/proc/sys/vm/dirty_ratio: disk write buffer size as percentage of system memory

Handle out of memory
/proc/sys/vm/vfs_cache_pressure
/proc/sys/vm/min_free_kbytes
/proc/sys/vm/drop_cache