Storage

Storage Considerations

Artifactory manages binaries in a filestore and their metadata in a relational database.

Each binary has a unique checksum calculated during the upload, and the checksum and its metadata (filename, creation dates, package metadata, etc.) are stored in the database. Read more on the checksum-based storage.

Storage Layers

There are two layers of storage Artifactory manages

  1. Local storage: The local disk on the server running Artifactory itself. Used for
    1. Local downloads cache. This uses the local node’s disk and not an external disk. In the case of Kubernetes, this is an emptyDir volume
    2. Eventual (not recommended) for temporary storage of uploaded binaries until they are fully uploaded to the final filestore
      1. IMPORTANT: Avoid using the eventual storage when possible with the various direct upload solutions like the S3 direct upload
      2. If you must, when running in Kubernetes, don’t use the eventual on the local node storage but rather an external PVC so on a pod rescheduling, the data is not lost
  2. Filestore: An external storage for the final storage of Artifactory binaries
    1. This should not be in the server’s local disk but rather a dedicated network storage like object storage or NFS

NOTE: If using NFS as Artifactory’s filestore, make sure it has high throughput and low latency. NFS is not recommended for high load environments.

Local Storage Considerations

The local disk on the server running Artifactory should be with the following specs:

  • Type: SSD
  • Size: 500GB for cache-fs, eventual and temp work
    • More storage used for the cache-fs will improve download performance
  • IOPS: 6000 or higher
  • Throughput: 1000 MB/sec or higher

Detailed specs for the local storage disks and performance can be found in the Sizing section of this guide.

Filestore Storage Considerations

There are many types of external storage that can be used for Artifactory as its Filestore. Some examples are: S3, EBS, etc. Following is a list of the common options to consider.

JFrog Artifactory supports object storage providers like AWS S3, Google Cloud Storage and Azure Blob storage for the final storage of its binaries. The main benefit is a very robust, reliable, scalable and high performance storage.

Artifactory can also harness these providers’ support for direct cloud storage download and direct cloud upload, which boosts performance and scale.

Follow JFrog instructions on setting up the preferred object storage for your Artifactory.

The Direct Upload mechanism enables you to upload without the need to maintain persistent storage for the eventual directory. This mechanism may also allow for faster uploads, since there is less contention for disk IO, particularly when Artifactory is hosted on AWS.

IMPORTANT: The object storage bucket must be in the same region your Artifactory is running in.

Filestore Sharding

JFrog Artifactory can use multiple storage volumes with sharding between them, so you gain better durability and availability of the stored binaries.

See instructions on setting up filestore sharding.

NFS

JFrog Artifactory can work with an external file storage like NFS or similar. Follow instructions for setting up nfs as filestore to use NFS as the filestore.

The NFS is basically mounted to the local server’s file system as a volume and Artifactory is using it like local filestore storage.

Required specs:

  • Sub ms to 5 ms latency
  • Size: 1TB (NFS mount size must have enough storage to host all your binaries)
  • IOPS: 6000 or higher
  • Throughput: 1000 MB/sec or higher

Filestore Sizing

JFrog Artifactory holds all of your organization’s binary assets. As such, it will require a lot of storage and ideally, such that it can grow as needed.

Cloud providers object storage like AWS S3 offer limitless storage.

It’s recommended to allocate multiple terabytes for a production environment to begin with.

Additional Information

See Best Practices for Managing Your Artifactory Filestore for more information on Artifactory storage best practices.