A HA scale out DDP consists of a Dual DDP Head with the Ardis Virtual File System AVFS for the metadata.
Data is stored on one or more DDP redundant storage arrays from the DDP EXR series in combination with one or more EXRAID, redundant RAID controllers.
The dual DDP Head with AVFS can also be combined with 3rd party and legacy storage array with up to 200 GbE/s bandwidth and using RoCE, FC, NVME-oF or Infiniband between desktops and HA DDP in non standard cases.
However standard access between the HA DDP and desktops is via iSCSI/AVFS using Ethernet. An example of this is shown in the two figures.
Scale out HA DDP with SSD Cache and spindles.
From 3 GB/s and up depending on SSD packs and EXRAID.
Ask us for other configurations and prices.
Scale out HA DDP with spindles.
Here a 3GB/s bandwidth and 3 PB capacity.
Ask us for other configurations and prices.
HIGH AVAILABILITY DDP
The tables show the specifications. Also third party redundant RAID arrays can be ordered.
|Dual DDP Head||Ex Raid|
|Content||dual MB each with dual 10GbE/RJ45, dual Xeon, 32GB||HA raid controller|
|Available PCIe slots||2 x 2||-|
|SSD4 or SSD8 packs||2 x 2 raid 1- SSDs hold the operating system||-|
|HD4 or HD8 packs||-||-|
|Ethernet cards||10GbE, 25GbE, 40GbE, 100GbE, 2PCIe slots each||2 x dual 10 GbE/SFP + on board|
|FC cards||on request||-|
|Infiniband cards||on request||-|
|Other cards||on request||-|
|DDP dimensions||42,8 x 73 x 25 cm / 45.3 x 28.8 x 3.5 inch||43,5 x 44,5 x 40 cm / 17.1 x 17.5 x 15.7 inch|
|DDP power usage||1280W/2 x dual power supplies||200W / dual power supplies|
|Package dimensions||115 x 62 x 25 cm / 45.3 x 24.4 x 9.8 inch||64,5 x 55,7 x 27,9 cm / 25.4 x 22 x 11 inch|
|Package weight||38,8 Kg / 85 lbs / base system with rails in carton, packed||28 Kg / 61.6 lbs|
|Remark||redundant HA AVFS Controller||For SAS to iSCSI|
HIGH AVAILABILITY DDP STORAGE ARRAYS
|Content||dual SAS expander||dual SAS expander||dual SAS expander||dual SAS expander|
|SSD4 or SSD8 packs||SSD8 up to 2; 0.96, 1.92, 3.80TB, SAS||SSD8 up to 3; 0.96, 1.92, 3.80TB, SAS||SSD8 up to 3; 0.96, 1.92, 3.80TB, SAS||SSD8 up to 10; 0.96, 1.92, 3.80TB, SAS|
|HD4 or HD8 packs||HD8: up to 2; 4, 6, 8, 10, 12, 14TB, SAS||HD8: up to 2; 4, 6, 8, 10, 12, 14TB, SAS||HD8: up to 2; 4, 6, 8, 10, 12, 14TB, SAS||HD8: up to 2; 4, 6, 8, 10, 12, 14TB, SAS|
|DDP dimensions||43 x 53,4 x 31,2 cm / 16.9 x 21 x 5.2 inch||43 x 53,4 x 17,4 cm / 16.9 x 21 x 6.9 inch||43 x 53,4 x 8,8 cm / 16.9 x 21 x 3.5 inch||43,4 x 97,5 x 17,6 cm / 17 x 38.4 x 7 inch|
|DDP power usage||549W / dual power supplies||549W / dual power supplies||549W / dual power supplies||1600W / dual power supplies|
|Package dimensions||59,6 x 78,5 x 44 cm / 23.5 x 30.9 x 17.3 inch||59 x 85,1 x 47,8 cm / 23.2 x 33.5 x 18.8 cm||59,5 x 78,5 x 37,8 cm / 23.4 x 30.9 x 14.9 inch||67,5 x 112,0 x 63,6 cm / 26.6 x 44.1 x 25.0 inch|
|Package weight||29,8 Kg / 65.7 lbs / base system with rails, pkg||34 Kg / 75 lbs / base system with rails, pkg||23 Kg / 50.7 lbs / base system with rails, pkg||67,67 Kg / 149.2 lbs / base system with rails, pkg|
|Remark||HA storage array||HA storage array||HA storage array||HA storage array|
Scaling in Capacity
The HA DDP can be expanded with storage arrays with SSD and HD packs of different capacities. The storage arrays can be added without DDP down time.
Scaling in Bandwidth
Desktops have simultaneous parallel access to all storage arrays. By adding storage arrays the bandwidth can increase in steps of more then 10GB/s., depending on the storage array added.
The HA AVFS file system is an Ethernet SAN system developed for Media and Entertainment ( M and E )
M & E needs uninterrupted high data rates for audio, video and film files over Ethernet. This can only be guaranteed with block IO (SAN) data access. Often in data centers the standard protocol for this is iSCSI. So iSCSI is a perfect choice.
To be able to share storage (shared storage server) a file system/metadata system was needed with the following requirements:
- Must be a single file system which independently can be scaled in performance and capacity
- Must have a small footprint and able to handle metadata fast with minimal overhead
- File system/metadata must be independent and separate from data containers
- Must have data access in parallel between clients and storage arrays
- Must be able to integrate with different types and brands of storage arrays
- Must be able to integrate SSD and HD packs of differing capacities for a competitive performance-price ratio
- Must have a directory/folder based access right and quota management system
- Must have file based SSD caching
- Must have the ability to move data between tiers transparent for clients
- Must have transparent data and/or metadata failover capability for writing and reading transparent for clients.
When Ardis Technologies realised that these requirements were not met in the file systems available on the market the decision was made to develop their own. Since more then 10 years now DDPs are delivered with AVFS on board and AVFS itself has evolved into what it currently offers including its file caching and high availability features.
HA DDP comes with HA AVFS on board. HA AVFS is managed by admin using the web interface. Although possible, there is no need for users to be confronted with the web interface.
HA AVFS is an in house development of Ardis Technologies. This has the advantage that issues can be solved without having to consult another party. A short reaction time is guaranteed this way. HA AVFS is the property of Ardis Technologies, manufacturer of the Dynamic Drive Pool, in short DDP series of storage servers
Yes. SNMP servers and other storage monitoring devices can be used.
Archiware P5 comes standard as part of AVFS. In a smaller setup P5 can be used to control a tape streamer or other SAS equipped library device. Or it can be used as client. When HA AVFS is used it is best to use a separate P5 server and mount HA AVFS ddp volume via iSCSI/AVFS.
AD, OD user names can be imported and be kept synchronized. Authentication is via AD or OD. Access rights and other attributes for these users are created and managed in AVFS.
HA AVFS runs on Linux and support SMB. It includes the standard up to date Linux SMB server, which internally in the Dual DDP Head is connected via iSCSI. When SMB is used both data and metadata is routed via the Dual DDP Head while iSCSI/AVFS access the data in parallel. When using SMB scale out and transparent data failover is not supported.
Yes but only installed on a dual DDP Head. A Dual DDP Head can be purchased in combination with capacity depending licensing costs for example to replace a third party file system in an existing situation with existing storage systems.
AVFS in combination with a Dual DDP Head can be used to cluster iSCSI, FC and Infiniband storage systems although the preferred protocol is iSCSI. The systems then function as data containers managed by AVFS.
HTTPS and two-factor authentication can be used for logon. AVFS file system remains in the DDP separate from the data and is never exposed to desktops. Operators only get to see a representation required for the Folder Volume mounted. AVFS and iSCSI can be physically separated using different network ports and subnets. Also hardware based hard disk encryption can be used when proper hard disks are used. AVFS is ISO 27001 and MPAA compliant. Auditing information is assembled during operation and does not need to be generated afterwards.
The play out should always go on. Therefore it is best to play out from SSD cache and cache material needed for play out using pinned so that it is available in time in the cache.
Ingest should continue uninterrupted no matter what with the media securely stored. Possible interruption can come from hard and software and hard disks issues. HA DDP assures that when there is hard or software failure ingest during a fail over continuous without hick up. In addition a Data Location can be optionally kept aside which can be used when one of the disks within a Data Location becomes too slow.
HA DDP fails over transparent for users including MAM servers. DDP uses a single file system with one directory/folder tree. If the MAM server connects with ddpvolume there are just (sub)folders to consider. AVFS also supports hard-links. Hard-links in combination with the single file system is ideal for MAM systems. Also without workflow interruption bandwidth and or capacity can be increased or maintenance can be done.
Almost all file systems have an ACL per file per user. AVFS uses ACLs per folder per user. For almost all activity in the M & E industry the property of a directory/folder is leading. Having ACL on directories/folders simplifies handling of access rights.
Various MAM applications benefit when a storage system supports hard-links. AVFS will support hard-links. Both hard-links and the single file system allow MAM and other applications to optimally use hard-links in their workflow.
AVFS is a single file system. It is separate from the storage arrays with the Data Locations it manages. A desktop has parallel access to these storage arrays of which its Data Locations behave
independently. So the total bandwidth is the sum of the bandwidth of the storage arrays. Also capacity can be increased separately. So yes, AVFS is a scale out file system.
Data access is in parallel using one or more Ethernet connections and as a standard using the iSCSI protocol.
On the desktop on Mac and in Linux volumes are displayed. On windows volumes are in My Computer. These volumes are folders on the file system, which have been given volume properties. These are therefore called Folder Volumes.
On the desktop on Mac and in Linux Volumes are shown. On windows Volumes are in My Computer. These Volumes are folders, which have been given Volume properties in the AVFS web interface, so called Folder Volumes.
A Folder Volume represents a folder on the file system. The data of files and subfolders and files displayed in the directory of such Folder Volume are on one of more Data Locations on the storage arrays.
These Data Locations are shown in Disk Utility on the Mac and in Computer Management on Windows.
Many systems with multiple nodes divide a file in blocks to spread over the nodes to balance the bandwidth. In that case all data must be present in the desktop before a file can be used. AVFS keeps a file within a Data Location boundary and a node can have multiple Data Locations. So also smaller AVFS setups can still benefit from load balancing. And to scale out storage nodes of different brands, capacities and manufacturing dates can be added.
AVFS uses file level caching. The caching technology called Dual Path is build-in in the AVFS file system. On playback a file is either coming from the cache or coming from the spindles. Or a complete file can be copied or ingested to the cache and duplicated to the spindles. If no longer in the cache the file is taken from the spindles. Caching, cache mode and direction are selectable per directory/folder.
The vast majority of systems use block level caching. There is no way of knowing to which file the data belongs. So the basic criteria, to keep data cached, is how often data blocks are used. Because parts of a file could be in the cache and other parts not, this could lead to unexpected play back interruptions. There is also no mechanism for file prioritization.
Yes. AVFS is an Ethernet SAN system with a single file system with one directory/folder tree. The available storage can be used up to 100 % and there is no pre-allocation. When more physical storage is added it automatically becomes available within AVFS.
One name space mostly describes a system which underneath combines multiple file systems. Such system has scaling, capacity and data movement limitations.
Single file system means that there is only one directory/folder tree giving access to all data no matter where it is. Data can be moved between Data Locations without changes to the directory/folder tree and transparent for the users.
With a SAN file system both data and the file system (metadata) are kept and managed separate. Data is stored as raw data in Data Locations. The single file system AVFS is elsewhere and holds information, on which Data Location the data can be found.
Unfortunately open source file systems such as XFS, ZFS and others cannot be easily tuned so that it can live up to M & E situations.
Ethernet is what everybody uses and is thus affordable. SCSI over Ethernet (iSCSI) is an established SAN protocol. Ethernet cards and switches are available up to 100 GbE/s bandwidth at affordable prices. Installing an Ethernet SAN system is easy nowadays.
To get the highest possible data bandwidth over a network the connection should not carry metadata. Or the amount of metadata should be controllable and as low as possible. This is possible when data and metadata can be handled separately. Only a SAN file system can guarantee this.
A HA DDP can be purchased through DDP distributors and dealers.
ICT companies, OEMs and integrators interested in purchasing a file system specially developed for M & E applications can E-mail us, go to Contact.
The HA AVFS file system in combination with the Dual Head (without DDP logo) is separately available.