Scalable Storage Clustering with redundancy? What to use?

[humbletech99]

I'm looking at implementing a large and scalable storage solution to replace lots of file servers which needs to meet the following requirements:

1. Redundancy
2. Scalability/Expandability
3. Performance
4. Single Directory tree for all storage

Ideally I'd like it to have something like a Global Filesystem like AFS or Microsoft's DFS where everything is organized into one directory tree, but also to have redundancy and data security like a Cluster Filesystem, so if one or more nodes breaks then it still works and we just put in new nodes. I also need to be able to extend it by putting in new nodes so that I can make the storage grow indefinitely.

At the moment my storage requirements are 15-30TB, but this will increase and so I need to able to add more space by adding more nodes.

What I think I am really after is a Cluster FileSystem, something like the Google File System, except that is not available cos Google are evil and eat up lots of talented open source people and keep cool stuff like that to themselves.

Ideally the solution should also be fast and reliable, so that I can crunch data on it from other servers.

Any ideas?

 

[xmsre]

Have you looked at NetApp? Clusered filers if you're getting new storage, of vFilers if you want to leverage existing SAN.

 

[humbletech99]

netapp look good, thats for the pointer.