Distributed File Systems Success Story

The Customer – HP Labs

HP Labs is the exploratory and advanced research group for Hewlett-Packard. The lab has some 600 researchers in seven locations throughout the world. HP Labs research areas include social computing, nanotechnology, sustainable IT, collaboration, business optimization, cloud computing, scale computing and information management.

The Challenge 

To design and develop a highly scalable distributed file system that could scale to more than 256 nodes.

The Solution 

Calsoft handled all the aspects of the project including:

  • Analysis
  • Scoping
  • Conceptual Design
  • Designing
  • Building
  • Testing

The Monster File System is a Cluster File System that allowed unmodified NFS clients and also specific clients to access a global file system that could support anywhere from 256 nodes upwards. This project had a lot of interesting research potential, especially in the area of scalability and distributed file system design; we designed & implemented this system on FreeBSD and Linux.

The Environment 
Technology  Distributed File Systems
Operating System FreeBSD, Linux
Debugging Tools Gdb, DDD
Skills FreeBSD, Linux Kernel internals
Languages C
The Benefits 
  • Increased acceptance of Network attached storage and the imminent appearance of Gigabit Ethernet
  • Metadata performance is a Bottleneck for most file systems
  • Location transparency:- DFS simplifies migrating data from one file server to another
  • Storage scalability:-Deploys additional or high performance file servers on the newer servers
  • Resource management and accessibility