Sending big data packets with ibr-dtn
Hi all,
I am trying to identify the main features of IBR-DTN overlay and what is going on beyond the basic tutorial in the website. I have initialized two nodes running in epidemic mode (change corresponding parameter in configuration file /etc/config/ibrdtn) and i tried to send a 5M file from one to another using dtninbox/dtnoutbox.
So, @sender_node : 1) root@OpenWrt:~/outboxFolder# dd if=/dev/zero of=output5M bs=5M count=1 1+0 records in 1+0 records out
2) root@OpenWrt:~# dtnoutbox outboxSender outboxFolder/ dtn://host1/inboxReceiver files: output5M Killed
@Receiver_node :
Tue Feb 11 10:33:49 2014 WARNING BundleCore: bundle rejected: bundle has expired ([0.66] dtn://host2/outboxSender) Tue Feb 11 10:33:49 2014 WARNING TCPConnection: bundle has been rejected: bundle is expired
and of course there was nothing at the inboxFolder when I typed: root@OpenWrt:/overlay# dtninbox inboxReceiver inboxFolder/
Moreover, in order to extend router's capabilities I have performed an external root and I changed the storage option in the configuration file:
##################################### # storage configuration # #####################################
config 'daemon' 'storage' # possible engines are: simple, sqlite option engine simple option blobs /tmp/ibrdtn/blobs # option bundles /tmp/ibrdtn/bundles option bundles /overlay/inboxFolder # option container /tmp/ibrdtn/container.img # option path /tmp/ibrdtn/container # option limit 1G
Is there a limit in how big are the data packets? Furthermore, is there any more extended tutorial where the parameters of the configuration file and how to change them according to our needs, are explained in more detail?
Thanks, Theodoros
Hello Theodoros.
Am 11.02.2014 11:43, schrieb Theodoros Bourchas:
Tue Feb 11 10:33:49 2014 WARNING BundleCore: bundle rejected: bundle has expired ([0.66] dtn://host2/outboxSender) Tue Feb 11 10:33:49 2014 WARNING TCPConnection: bundle has been rejected: bundle is expired
The bundle protocol requires that the clocks of all nodes are in sync and have a valid timestamp (> 2000-01-01 00:00:00). Fix that and it will work.
Is there a limit in how big are the data packets?
Yes. The protocol limits the size to 2^64 bytes. Additional your platform may limit the file sizes on 32-bit systems this is usually 2^32 byte.
Furthermore, is there any more extended tutorial where the parameters of the configuration file and how to change them according to our needs, are explained in more detail?
There exists the configuration file for the standard dtnd without the UCI wrapper for OpenWrt. This file shows more details on the settings, but does not match to /etc/config/ibrdtn. https://github.com/ibrdtn/ibrdtn/blob/master/ibrdtn/daemon/etc/ibrdtnd.conf
In any case you should first know what you want and then check the configuration options to achieve it.
Kind regards, Johannes Morgenroth
In /etc/config/ibrdtn , the only thing that I have changed is the routing mode (epidemic), the hostname and the network interface in order to point at wlan0. I initialize the daemons with: dtnd -c /tmp/ibrdtn.config --badclock.
***************** root@OpenWrt:~# dtnd -c /tmp/ibrdtn.config Tue Feb 11 13:40:45 2014 INFO Configuration: Configuration: /tmp/ibrdtn.config Tue Feb 11 13:40:45 2014 INFO NativeDaemon: IBR-DTN daemon 0.10.2 (build 1d89427 ) Tue Feb 11 13:40:45 2014 INFO Configuration: Configuration: /tmp/ibrdtn.config Tue Feb 11 13:40:45 2014 INFO BundleCore: Local node name: dtn://host1 Tue Feb 11 13:40:45 2014 INFO BundleCore: Forwarding of bundles enabled. Tue Feb 11 13:40:45 2014 INFO NativeDaemon: using simple bundle storage in /tmp/ibrdtn/bundles Tue Feb 11 13:40:45 2014 INFO NativeDaemon: using BLOB path: /tmp/ibrdtn/blobs Tue Feb 11 13:40:45 2014 INFO SimpleBundleStorage: 0 Bundles restored. Tue Feb 11 13:40:45 2014 INFO NativeDaemon: API initialized using tcp socket: loopback:4550 Tue Feb 11 13:40:45 2014 INFO NativeDaemon: TCP ConvergenceLayer added on wlan0: 4556 Tue Feb 11 13:40:45 2014 INFO DiscoveryAgent: listen to [ff02::142]:4551 Tue Feb 11 13:40:45 2014 INFO DiscoveryAgent: listen to [224.0.0.142]:4551 Tue Feb 11 13:40:45 2014 INFO DiscoveryAgent: add interface wlan0 Tue Feb 11 13:40:45 2014 INFO DHTNameService: DHT initialized on Port: 9999 with ID: Tue Feb 11 13:40:45 2014 ERROR IPNDAgent: join failed on wlan0; 99: setsockopt() Tue Feb 11 13:40:45 2014 INFO NativeDaemon: Using epidemic routing extensions Tue Feb 11 13:40:45 2014 INFO EpidemicRoutingExtension: Initializing epidemic routing module
I don't know if I should do sth else to synchronize the nodes. After that when I send/receive files no messages appear in the daemon like the ones that I have seen in mailing list :
- 1.bundle received
*>* 2.queue bundle event *>* 3.bundle event *>* 4.transfercompleted event *>* 5.bundle event:bundle forwarded*
*But still packages less than 4Mbyte are retrieved (even without explicit notification).*
*Do I miss sth?*
*Best,*
*Theodoros*
On Tue, Feb 11, 2014 at 1:12 PM, Johannes Morgenroth < morgenroth@ibr.cs.tu-bs.de> wrote:
Hello Theodoros.
Am 11.02.2014 11:43, schrieb Theodoros Bourchas:
Tue Feb 11 10:33:49 2014 WARNING BundleCore: bundle rejected: bundle has expired ([0.66] dtn://host2/outboxSender) Tue Feb 11 10:33:49 2014 WARNING TCPConnection: bundle has been rejected: bundle is expired
The bundle protocol requires that the clocks of all nodes are in sync and have a valid timestamp (> 2000-01-01 00:00:00). Fix that and it will work.
Is there a limit in how big are the data packets?
Yes. The protocol limits the size to 2^64 bytes. Additional your platform may limit the file sizes on 32-bit systems this is usually 2^32 byte.
Furthermore, is there any more extended tutorial where the parameters of the configuration file and how to change them according to our needs, are explained in more detail?
There exists the configuration file for the standard dtnd without the UCI wrapper for OpenWrt. This file shows more details on the settings, but does not match to /etc/config/ibrdtn. https://github.com/ibrdtn/ibrdtn/blob/master/ibrdtn/daemon/etc/ibrdtnd.conf
In any case you should first know what you want and then check the configuration options to achieve it.
Kind regards, Johannes Morgenroth
-- Johannes Morgenroth Institut fuer Betriebssysteme und Rechnerverbund Tel.: +49-531-391-3249 Muehlenpfordtstrasse 23 Fax.: +49-531-391-5936 TU Braunschweig D-38106 Braunschweig
Am 11.02.2014 14:53, schrieb Theodoros Bourchas:
/But still packages less than 4Mbyte are retrieved (even without explicit notification)./ /Do I miss sth?/ /Best,/ /Theodoros/
Maybe you have not sufficient RAM to prepare the bundle in memory. This is the default since the tool can not know where to put temporary files. Thus, you have to define a temporary working directory for dtnoutbox -w /overlay/tmp [...]. Additional to that you should set the blobs path of the daemon to another file path with enough space.
Also consider to change your daemon.storage.bundles path to something else. The name of the folder indicates that this is used as an application inbox. This would be a fatal error. The bundles directory is the place where the daemon (and only the daemon) stores and access bundles. Working directories of the daemon should never overlap with folders of applications.
Kind regards, Johannes Morgenroth
I will try changing these parameters, but I still never receive in the daemons running, messages like these: https://mail.ibr.cs.tu-bs.de/pipermail/ibr-dtn/2013-February/000305.html
NOTICE option is enabled somehow?
Thanks, Theodoros
On Tue, Feb 11, 2014 at 3:06 PM, Johannes Morgenroth < morgenroth@ibr.cs.tu-bs.de> wrote:
Am 11.02.2014 14:53, schrieb Theodoros Bourchas:
*But still packages less than 4Mbyte are retrieved (even without explicit notification).*
*Do I miss sth?*
*Best,*
*Theodoros*
Maybe you have not sufficient RAM to prepare the bundle in memory. This is the default since the tool can not know where to put temporary files. Thus, you have to define a temporary working directory for dtnoutbox -w /overlay/tmp [...]. Additional to that you should set the blobs path of the daemon to another file path with enough space.
Also consider to change your daemon.storage.bundles path to something else. The name of the folder indicates that this is used as an application inbox. This would be a fatal error. The bundles directory is the place where the daemon (and only the daemon) stores and access bundles. Working directories of the daemon should never overlap with folders of applications.
Kind regards, Johannes Morgenroth
-- Johannes Morgenroth Institut fuer Betriebssysteme und Rechnerverbund Tel.: +49-531-391-3249 Muehlenpfordtstrasse 23 Fax.: +49-531-391-5936 TU Braunschweig D-38106 Braunschweig
Am 11.02.2014 15:24, schrieb Theodoros Bourchas:
I will try changing these parameters, but I still never receive in the daemons running, messages like these: https://mail.ibr.cs.tu-bs.de/pipermail/ibr-dtn/2013-February/000305.html
NOTICE option is enabled somehow?
If you enable the logfile options in /etc/config/ibrdtn and start the daemon with /etc/init.d/ibrdtn start, the output will be in the log files. If you start the daemon manually you have to enable the NOTICE messages with "-v" to see them (see dtnd -h for more options).
$ dtnd -h IBR-DTN version: 0.10.2 Syntax: dtnd [options] -h|--help display this text -c <file> set a configuration file -D daemonize the process -k stop the running daemon -p <file> store the pid in this pidfile -i <interface> interface to bind on (e.g. eth0) -d <level> enable debugging and set a verbose level -q enables the quiet mode (no logging to the console) -t <threads> specify a number of threads for parallel event processing -v be verbose - show NOTICE log messages --version show version and exit --noapi disable API module --nodiscovery disable discovery module --timestamp enables timestamps for logging instead of datetime values
Kind regards, Johannes Morgenroth
Ok it seems that blobs and bundles directory needed more space. I have set in both my nodes the storage path to /overlay/TMP/bundles and /overlay/TMP/blobs. I tried to send a 5M file.
*@Sender_side*: root@OpenWrt:~/outboxfolder# tar -cf - output5M | dtnsend dtn://host1/inboxRe ceiver Transfer stdin to dtn://host1/inboxReceiver ************************************ root@OpenWrt:/# dtnd -v -c /tmp/ibrdtn.config --badclock Tue Feb 11 14:49:04 2014 INFO Configuration: Configuration: /tmp/ibrdtn.config Tue Feb 11 14:49:04 2014 INFO NativeDaemon: IBR-DTN daemon 0.10.2 (build 1d89427) Tue Feb 11 14:49:04 2014 INFO Configuration: Configuration: /tmp/ibrdtn.config Tue Feb 11 14:49:04 2014 INFO BundleCore: Local node name: dtn://host3 Tue Feb 11 14:49:04 2014 INFO BundleCore: Forwarding of bundles enabled. Tue Feb 11 14:49:04 2014 INFO NativeDaemon: using simple bundle storage in /overlay/TMP/bundles Tue Feb 11 14:49:04 2014 INFO NativeDaemon: using BLOB path: /overlay/TMP/blobs Tue Feb 11 14:49:04 2014 INFO SimpleBundleStorage: 0 Bundles restored. Tue Feb 11 14:49:04 2014 INFO NativeDaemon: API initialized using tcp socket: loopback:4550 Tue Feb 11 14:49:04 2014 INFO NativeDaemon: TCP ConvergenceLayer added on wlan0:4556 Tue Feb 11 14:49:04 2014 INFO DiscoveryAgent: listen to [ff02::142]:4551 Tue Feb 11 14:49:04 2014 INFO DiscoveryAgent: listen to [224.0.0.142]:4551 Tue Feb 11 14:49:04 2014 INFO DiscoveryAgent: add interface wlan0 Tue Feb 11 14:49:04 2014 INFO DHTNameService: DHT initialized on Port: 9999 with ID: Tue Feb 11 14:49:04 2014 ERROR IPNDAgent: join failed on wlan0; 99: setsockopt() Tue Feb 11 14:49:04 2014 INFO NativeDaemon: Using epidemic routing extensions Tue Feb 11 14:49:04 2014 INFO EpidemicRoutingExtension: Initializing epidemic routing module Tue Feb 11 14:49:05 2014 WARNING DHTNameService: bootstrapping from default domain failed with error: -2 Tue Feb 11 14:49:06 2014 WARNING DHTNameService: bootstrapping from default domain failed with error: -2 Tue Feb 11 14:49:13 2014 NOTICE NodeEvent: Node dtn://host1 available Tue Feb 11 14:49:51 2014 NOTICE BundleReceivedEvent: Bundle received [0.1] dtn://host3/YigNFjYqIsyFMIuS (local) Tue Feb 11 14:49:51 2014 NOTICE QueueBundleEvent: New bundle queued [0.1] dtn://host3/YigNFjYqIsyFMIuS Tue Feb 11 14:49:51 2014 NOTICE BundleEvent: bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS received Tue Feb 11 14:49:51 2014 NOTICE ConnectionEvent: connection setup dtn://host1 Tue Feb 11 14:49:51 2014 NOTICE ConnectionEvent: connection up dtn://host1 Tue Feb 11 14:50:17 2014 NOTICE BundleCore: singleton bundle delivered: [0.1] dtn://host3/YigNFjYqIsyFMIuS Tue Feb 11 14:50:17 2014 NOTICE TransferCompletedEvent: transfer of bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS to dtn://host1 completed Tue Feb 11 14:50:17 2014 NOTICE BundlePurgeEvent: purging bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS Tue Feb 11 14:50:17 2014 NOTICE BundleEvent: bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS deleted Tue Feb 11 14:50:17 2014 NOTICE BundleEvent: bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS forwarded
*@Receiver_side* root@OpenWrt:/etc/config# dtnd -v -c /tmp/ibrdtn.config --badclock Mon Feb 10 16:07:25 2014 INFO Configuration: Configuration: /tmp/ibrdtn.config Mon Feb 10 16:07:25 2014 INFO NativeDaemon: IBR-DTN daemon 0.10.2 (build 1d89427) Mon Feb 10 16:07:25 2014 INFO Configuration: Configuration: /tmp/ibrdtn.config Mon Feb 10 16:07:25 2014 INFO BundleCore: Local node name: dtn://host1 Mon Feb 10 16:07:25 2014 INFO BundleCore: Forwarding of bundles enabled. Mon Feb 10 16:07:25 2014 INFO NativeDaemon: using simple bundle storage in /overlay/TMP/bundles Mon Feb 10 16:07:25 2014 INFO NativeDaemon: using BLOB path: /overlay/TMP/blobs Mon Feb 10 16:07:25 2014 INFO SimpleBundleStorage: 0 Bundles restored. Mon Feb 10 16:07:25 2014 INFO NativeDaemon: API initialized using tcp socket: loopback:4550 Mon Feb 10 16:07:25 2014 INFO NativeDaemon: TCP ConvergenceLayer added on wlan0:4556 Mon Feb 10 16:07:25 2014 INFO DiscoveryAgent: listen to [ff02::142]:4551 Mon Feb 10 16:07:25 2014 INFO DiscoveryAgent: listen to [224.0.0.142]:4551 Mon Feb 10 16:07:25 2014 INFO DiscoveryAgent: add interface wlan0 Mon Feb 10 16:07:25 2014 INFO DHTNameService: DHT initialized on Port: 9999 with ID: Mon Feb 10 16:07:25 2014 ERROR IPNDAgent: join failed on wlan0; 99: setsockopt() Mon Feb 10 16:07:25 2014 INFO NativeDaemon: Using epidemic routing extensions Mon Feb 10 16:07:25 2014 INFO EpidemicRoutingExtension: Initializing epidemic routing module Mon Feb 10 16:07:26 2014 NOTICE NodeEvent: Node dtn://host3 available Mon Feb 10 16:08:04 2014 NOTICE ConnectionEvent: connection up dtn://host3 Mon Feb 10 16:08:31 2014 NOTICE BundleReceivedEvent: Bundle received [0.1] dtn://host3/YigNFjYqIsyFMIuS from dtn://host3 Mon Feb 10 16:08:31 2014 NOTICE QueueBundleEvent: New bundle queued [0.1] dtn://host3/YigNFjYqIsyFMIuS Mon Feb 10 16:08:31 2014 NOTICE BundleEvent: bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS received Mon Feb 10 16:08:54 2014 NOTICE BundleEvent: bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS delivered Mon Feb 10 16:08:54 2014 NOTICE BundlePurgeEvent: purging bundle [0.1] dtn://host3/YigNFjYqIsyFMIuS
************************** root@OpenWrt:~# dtninbox inboxReceiver inboxFolder/ Segmentation fault
While I was able to retrieve a 4M file, I am stuck with this segmentation fault in 5M file. What do u think is going wrong?
Thanks, Theodoros
On Tue, Feb 11, 2014 at 3:43 PM, Johannes Morgenroth < morgenroth@ibr.cs.tu-bs.de> wrote:
Am 11.02.2014 15:24, schrieb Theodoros Bourchas:
I will try changing these parameters, but I still never receive in the daemons running, messages like these: https://mail.ibr.cs.tu-bs.de/pipermail/ibr-dtn/2013-February/000305.html
NOTICE option is enabled somehow?
If you enable the logfile options in /etc/config/ibrdtn and start the daemon with /etc/init.d/ibrdtn start, the output will be in the log files. If you start the daemon manually you have to enable the NOTICE messages with "-v" to see them (see dtnd -h for more options).
$ dtnd -h IBR-DTN version: 0.10.2 Syntax: dtnd [options] -h|--help display this text -c <file> set a configuration file -D daemonize the process -k stop the running daemon -p <file> store the pid in this pidfile -i <interface> interface to bind on (e.g. eth0) -d <level> enable debugging and set a verbose level -q enables the quiet mode (no logging to the console) -t <threads> specify a number of threads for parallel event processing -v be verbose - show NOTICE log messages --version show version and exit --noapi disable API module --nodiscovery disable discovery module --timestamp enables timestamps for logging instead of datetime values
Kind regards, Johannes Morgenroth
-- Johannes Morgenroth Institut fuer Betriebssysteme und Rechnerverbund Tel.: +49-531-391-3249 Muehlenpfordtstrasse 23 Fax.: +49-531-391-5936 TU Braunschweig D-38106 Braunschweig
Am 11.02.2014 15:58, schrieb Theodoros Bourchas:
root@OpenWrt:~# dtninbox inboxReceiver inboxFolder/ Segmentation fault
While I was able to retrieve a 4M file, I am stuck with this segmentation fault in 5M file. What do u think is going wrong?
The same as for dtnoutbox: Specify a working directory!
dtninbox -w /overlay/inbox_tmp [...]
$ dtninbox -h -- dtninbox (IBR-DTN) -- Syntax: dtninbox [options] <name> <inbox> <name> the application name <inbox> directory where incoming files should be placed * optional parameters * -h|--help display this text -w|--workdir temporary work directory
Regards, Johannes Morgenroth
participants (2)
-
Johannes Morgenroth -
Theodoros Bourchas