Reference no: EM131910477

Discuss how breaking the traditional RAID concepts helps Big Data deal with ever-growing needs of a storage system.

Data and storage expansion is a challenging task for cloud service providers and data center managers. New technologies can be leveraged to implement storage back-ends, but when you have as much data growth as Google, sometimes you need to think out of the box. The Google File System uses many architecture concepts as RAID, but pushes that up further into the OSI reference model by using software defined storage. Have you ever wondered how all that cloud storage was being presented to end users? A lot of that is proprietary, but it is rooted in the basic concepts of RAID, distributed file systems and clustering. If we dig a little we can find that Google cloud services offers some details that does give us some insight into the google file system (GFS). "Google hasn't done anything unique with their platform that other people had not already. They just put it together and scaled it to unprecedented heights....GFS doesn't use any specific RAID controllers, fibre channel, iSCSI, HBAs, FC or SCSI disks, dual-porting or any of the other costly hardware we expect in a wide-awake data center. And yet it all works and works very well. (Harris, n.d)" By having a software data manager take care of this block storage and having multiple replication points, GFS is not bound by RAID structures and configurations. Google does use clustering and data chunking of their disks with redundant copies at the block layer instead of using the predefined structure of RAID arrays.