What's New
The Spectra Vail application is updated to Vail 3.5.0, which brings with it the following new features:
Current Version Storage
You are now able to use non-versioned buckets for cloud storage. This storage is identified as "Current-Version Only" storage, and can only be used for the current version of an object. Since objects disappear when a new version is created, Current Version storage is not allowed to be the only storage location of an object.
Storage Groups
Multiple storage definitions can be grouped and used as a single destination. In a Storage Group, data is distributed across the members of the group. This can significantly improve performance, and allows for expanded options when using multiple endpoints.
Application Database Improvements
The PostgreSQL database used by the Vail application now stores all partition data in a separate table, and all local user and user groups use a separate partition for each account. This simplifies indexes, reduces transaction overhead and storage requirements, and improves performance.
| Note: | This behavior only applies to new installations of Vail 3.5, or later. Older Vail installations that upgrade to Vail 3.5 may manually convert to the new database behavior. Contact Spectra Logic Technical Support for assistance. |
Use With Buckets
The linked-bucket behavior used in previous Vail versions changes in Vail 3.5. The link bucket behavior has now been separated into a "Use With Buckets" feature and an "Import Content" feature. The previous linked bucket functionality is now achieved by selecting a single bucket to use, and selecting "Import Content". The "Use With Buckets" option allows you to make storage exclusive to a particular bucket or buckets, so objects from other buckets can never use it. This prevents storing inappropriate content, and also allows for faster deletion of the storage.
One use case of the previous linked bucket behavior was as a way to change the path for objects created on a cloud bucket. By default, the path on cloud storage includes the bucket name. Since linked bucket storage can only be used with the linked bucket, the bucket name is not included in the cloud bucket path. Now you can uncheck "Import Content" and configure a single "Use With Buckets" entry. When cloud storage can only be used with a single bucket, the cloud bucket path doesn't include the Vail bucket name. The cloud bucket path is now exposed as a modifiable value using the API. It cannot be changed using the Vail management console.
This change also makes it possible to modify the linked bucket attributes of storage. You can modify the "Import Content" checkbox to add this to existing storage. You cannot make storage more restrictive, but you can select additional "Use With Buckets" or remove the list entirely to allow previously restricted storage to be used by all buckets.
Performance Improvements
Lifecycle processing performance is significantly improved. The Vail application now uses more simultaneous operations and adjusts quickly to a large backlog of pending work. On start up, the Vail application no longer waits for initialization of the pending lifecycle queues before completing other initialization. Lifecycle queue initialization completes as a background task, and lifecycle processing for each storage destination begins after that initialization completes.
Volume Storage Write Throttling
You can now configure a write rate limit for a volume storage. This limit is the number of MiB per second the volume storage is allowed to process once the storage usage caution threshold has been reached. If storage usage is below the caution threshold, writes are unlimited. The configured rate limit is enforced at the caution threshold, and the rate is then reduced linearly as the volume usage increases. The rate limit applies to lifecycle processing and all data writes to the given storage.
Shared Packs for Efficient Tape Storage
The default maximum shared pack size is now 4 GiB. This allows for less fragmented space on tape storage as a smaller shared pack size increases the likelihood that all objects in the pack are deleted, allowing the pack itself to be deleted. Additionally, you can now configure the maximum shared pack size on tape storage.
Maximum Part and Object size
Starting with Vail 3.5.0, the maximum upload part size supported by the Vail application is 100 GiB, with a maximum object size of 1 PiB (10,000 parts of 100 GiB each). Not all cloud providers allow for objects of this size. An error is generated if an attempt is made to add cloud storage to an object that is larger than what the storage supports.
Automatic Update of Default BlackPearl Data Policies
The Vail application is updated to simplify the creation of tape storage. When you register a BlackPearl endpoint, the Vail application now updates the default data policies on the BlackPearl system for compatibility. The default data policies are updated to use Object Naming and CRC-32C checksums. This update only occurs if the default data policies are not already in use. Other data policies are not affected.
Setting Cloud Storage Class
When creating cloud storage, the Storage Class assigned to created cloud objects is now explicitly selected instead of being determined by the storage class of the storage definition. Additional cloud storage tier definitions for Microsoft Azure and Google Cloud storage have been added to the Vail application.
New Storage Class Compatibility
The Vail application now accepts FSX_OPENZFS as a valid storage class.
S3 Compatible Link Notifications
Linked bucket notifications are now supported on S3 compatible storage in addition to AWS linked buckets. The service must be configured to send SNS notifications in the required format.
Account Connection Limit
A connection limit can now be configured for a user account to prevent single account from using all available connections.
Default Lifecycle
After upgrading to Vail 3.5, all buckets must have an assigned lifecycle. If a lifecycle is not currently assigned to a bucket, the bucket owner's default is used. If the bucket owner does not have a configured default lifecycle, the global default is used. When updating to Vail 3.5, a new default lifecycle is created with basic settings that delete pending multi-part uploads and removes useless delete markers.
You can set any lifecycle as the default and can delete the automatically created default.
Clone Operations By Prefix
You can now create and delete clones for all objects using a provided prefix. Additionally, you can create restore requests for all objects with a shared prefix. This functionality is only available through the management API.
S3 Checksum and Signature Support Improvements
The Vail application now offers additional support for AWS checksum handling and signature Version 4a.