How to scale InfluxDB in docker service with persistent data

Question

What have I done?

Got docker installed, version details below:

<div id="submitsearch">
   <input type="submit" value="Google Search">
   <input type="submit" value="I'm feeling lucky">
</div>

Docker running in swarm mode. Started a influxdb docker service by issuing following command.

Server:
 Version:      17.11.0-ce
 API version:  1.34 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   1caf76c
 Built:        Mon Nov 20 18:36:09 2017
 OS/Arch:      linux/amd64
 Experimental: false

Then, I created a dummy database in influxdb and inserted dummy data in it. Getting a count from influxdb is consistent as its a single service and single datasource (mounted volume). All good.

Scale up

Scaled up the influxdb service to run 5 instances by using following command.

# create network
docker network create influxdb-network --driver overlay
# create the service
docker service create --name 'influxdb' --network 'influxdb-network' --mount type=volume,source=influxdb-volume,destination=/var/lib/influxdb influxdb:1.4

Now when I insert new data it get Load balanced and inserted to all the five instances separately. How to have a consistent data across all 5 instances of influxdb?

Answer 1

Docker volumes are tightly coupled to the machine where they are created, thus they don't scale for a distributed application. This is still a lacking feature in docker ro have a distributed storage backend. You can workaround this is issue by having a network storage such as NFS where the volume stores it data.

There are some projects that try to deal properly with this problem. Most notably RexRay and tyhe discontinued project Flocker.

In conclusion, when it comes to scaling databases, docker won't help much. You need to deal with scaling the database the way you would without having Docker.