Setting up DRBD with drbdsetup

Let's assume that your two machines are named 'node1' and 'node2', and that you want to use /dev/hdc6 as the lower level block device on both. The following commands are needed on node1 to setup drbd:

insmod drbd.o
drbdsetup /dev/nb0 /dev/hdc6 B node1 node2

insmod drbd.o
drbdsetup /dev/nb0 /dev/hdc6 B node2 node1

When loading the module you may pass a parameter to the module. The name of the parameter is minor_count and with it you specify the number of DRBD devices. Its default is 2, thus you can use /dev/nb0 and /dev/nb1 per default.

The B in the command line of drbdsetup selects the protocol.

protocol	description
A	A write operation is complete as soon as the data is written to disk and sent to the network.
B	A write operation is complete as soon as a reception acknowledgement arrives.
C	A write operation is complete as soon as a write acknowledgement arrives.

If you want to learn more about the protocols, have a look at the paper in the download section.

There are a number of options you may append to the above commands.

option name	short	arg	description	default
--timeout	-t	y	If communication blocks for val * 1/10 seconds, drbd falls back into unconnected operation.	30
--sync-rate	-r	y	The synchronisation sends up to val KB per sec.	250
--skip-sync	-k	n	Instructs drbd not to do synchronisation.
--tl-size	-s	y	Sets the size of the transfer log(=TL). The TL is used for dependency analysis. For long latency high bandwith links it might be necessary to set the size bigger than 256. You will find error messages in the system log if the TL is too small.	256
--disk-size	-d	y	Sets drbd's size. When set to 0, drbd negotiates the size with the remote node.	0
--do-panic	-p	n	Drbd will trigger a kernel panic if there is an IO error on the lower_device. May be useful when drbd is used in a HA cluster. Do not forget to set the reboot after panic counter to a positive value. E.g. echo 1 > /proc/sys/kernel/panic, sets it to one second.
--connect-int	-c	y	If DRBD cannot establish a connection, it will retry to set up a connection every val seconds. You can disable this by setting the option to 0.	10
--ping-int	-i	y	A secondary cannot distinguish a broken link from an idle link unless you allow this feature. DRBD will send a no-op packet over the link if it is idle for more than val seconds. To disable this feature set it to 0.	10

And finally your cluster membership service needs to tell one of the two to become primary by running:

drbdsetup /dev/nb0 PRI

You have to take care that you never assign the primary role to both nodes. If you do that, drbd will disconnect and return to Unconfigured state immediately. If you want to migrate a service while both nodes are up, you have to bring both nodes into secondary state before assigning the primary role to another node. You can bring a node into secondary state by running:

drbdsetup /dev/nb0 SEC

When a node rejoins the cluster, its device must be brought to the new state. By default drbd assumes that the node still has its old disk and that it is sufficient to update only the blocks that have been modified in the absence of the other node. It might be of interest to your cluster manager when this synchronisation process is finished.

drbdsetup /dev/nb0 WAIT

This command returns as soon as any synchronisation process (SyncingAll or SyncingQuick) is finished. If there is no synchronisation process running, it returns immediately. You can use this on the primary and on the secondary node.

In case you had to replace the disk you need to trigger complete synchronisation by running the following command on the primary node:

drbdsetup /dev/nb0 REPL

To get rid of an device you can use:

drbdsetup /dev/nb0 DOWN

You can use cat /proc/drbd to get information about drbd's state. Example:

version       : 57

0: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 of:0
1: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 of:0

Each line shows one device. E.g. The Line starting with 0: is about /dev/nb0.

Your application (most probably a filesystem) may access the device /dev/nb0 in read-write mode if drbd is in primary state. Otherwise only read-only access is permitted.