Storage Types
Overview
firstcolo Cloud offers two different types of block storage for instances: Persistent volumes and local storage.
| Storage Type | Is Persistent? | Supports Backups? | Survives hardware failures? |
|---|---|---|---|
| Volume Storage | Yes | Yes, saved to object store | Yes, instance can be evacuated without data loss |
| Local Storage | No, will be deleted with the instance | No, but supports exports | No, instance will be recreated from image |
Persistent Volumes
Volumes are handled by the Cinder Block Storage service and stored on a distributed cluster running Ceph. They offer high performance, redundancy and scalability at the cost of latency, as connected via network. Because the data is stored on a distributed cluster, it does not rely on a single host. Therefore, instances booted from volumes can be restarted on another compute node in case of a hardware failure.
Note
If creating an instance via the Dashboard (see first steps tutorial), it will create a persistent boot volume by default.
Cinder has an integrated backup functionality which stores volume snapshots in an object store bucket. Because object store in firstcolo Cloud is driven by Swift, the objects are not stored in Ceph, but on a dedicated storage cluster.
Use cases
Volumes can be used as boot medium, as well as for data-only storage. If used for data-only, a volume can be attached and detached from instances even during runtime.
Therefore, volumes can be used for storing data persistently and independent of the instance as well as providing more storage capacity. Instances without automated provisioning should use volumes as boot medium to prevent data-loss and ensure availability in case of a hardware failure.
Local Storage
Local storage is handled by the Nova Compute service which stores the data directly on SSDs builtin to the compute nodes. This offers low latencies, but the storage is considered as ephemeral and will be deleted with the instance it is connected to. Further, in case of a hardware failure the data cannot be recovered, the instance will be recreated using its backing image and all configuration that is not injected via cloud-init will be lost.
Warning
Unlike the Dashboard, tools which use the OpenStack APIs may create instances using local storage by default. For how to use persistent volume instead checkout Terraform - Boot from Volume for example.
Because local storage is ephemeral by design, there is no integrated backup functionality. Altough, it is possible to create an image from an instance with local storage. This process is called shelving and converts the local disk before uploading it to the image service.
Note
Instance shelving should not be used for creating backups as the conversion to an image is expensive and may fail with bigger instances.
Use cases
Local storage can be used as boot medium, but should only be considered if the instance configuration is handled by cloud-init or some kind of automation and all data can be recreated easily or is stored on separate volumes.
Therefore, local storage is ideal for clustered services which withstand an instance failure and recover automatically, like database clusters or cloud-native applications (checkout anti-affinity-groups to ensure that instances are not placed on the same compute node). Kubernetes worker nodes which are managed by a cloud controller (like within firstkube) are another example.
Questions & Answers
How do I use local SSD storage?
For a quick start to evaluate or just play with local SSD storage, please see our Tutorials.
When should I better not use local SSD storage?
Traditional single server setups often suffer from performance penalty when running on distributed storage. While it may be tempting to build them on local storage to gain speed, we discourage to use it this way because of the lower redundancy and availability. To put it into simple words, you put your service and your data at risk, when running a single server setup on local storage.
Also, the provisioning (e.g. installation of packages and configuration) of the instance should happen automatically. Otherwise your service will be unavailable after a host evaluation, even if all application data is stored in persistent volumes.
What can I do to make my setup suitable for local SSD storage?
Depending on your application, there are many proven ways to design either your setup or your application in a way that provides for the missing redundancy. Application servers can be multiplied and put behind a load balancer, databases can be set up in replication topologies or clusters. Please contact us if you need further assistance.
Can I use local SSD storage for volumes?
No, local SSD storage is only available as ephemeral storage.
Can I combine local SSD storage with distributed storage?
Yes. You can attach volumes with distributed storage to your instances. They will appear as additional block storage devices in your operating system.
Can I replace local SSD storage with a volume?
Yes, it is possible to convert the local boot disk of an instance to a volume, see our howto for more information.
Can I use local SSD storage in other sizes?
Local SSD storage is exclusively available in the sizes defined in our flavors.
Can I resize local SSD storage instances?
No, local storage instances cannot be resized, you have to either recreate them with a different size and migrate your data or upgrade your flavor to a larger one. You can, however, attach a volume, move your data onto it, then detach it and attach it to another system where you move your data to the local SSD storage. Depending on your application, there are likely smarter ways to bring new nodes into the cluster and seed them, e.g. promoting a replication slave.
What happens in case of hardware failures?
Local SSD storage instances can only be moved between hypervisors if the hypervisor, especially the hardware raid, is fully functional. That means in case of a hardware raid failure where the compute node is not accessible anymore, we cannot migrate the virtual machine to another compute hypervisor to restore availability. The instance will be evacuated by us at some point, which means that it will be recreated from the image on a new hypervisor.
The data which was saved on the local disk will be lost.
What about hardware maintenance?
We need to regularly reboot our compute nodes for maintenance. Before we start the maintenance all virtual machines will be migrated to a different compute hypervisor.
How many nodes/instances will be affected by local SSD storage node maintenance simultaneously?
Planned maintenances will only affect one compute node at a time and between two maintenances there will be half an hour of recreation to allow the affected systems to re-join their clusters or whatever. It will, however, affect all local SSD storage instances on the same compute node. To ensure, that redundant systems will not be affected simultaneously, you must put them into anti-affinity-groups.