Storage in AWS

Storage in AWS




There are 3 major storage type in AWS :

  • Block storage (Elastic Block Storage - EBS)
  • File storage (Elastic File Storage - EFS)
  • Object storage (S3 bucket)
HDD and SSD are the main storage units for all these three. 

Block storage - HDD and SSD are the devices that uses blocks to store information inside. Blocks are small fragment/cluster of data. Basically AWS provides this so that we can partition and format it easily. They are ideal for applications that require low-latency access to data, such as databases and enterprise applications.

File Storage - Is different than block storage, while under the hood it depends on block based storage units, it is connected to internet so that we upload/download files over internet. SSD and/or HDD units are inserted Network Attached Storage Servers so that is how it works. It is ideal for content management systems, web serving, and big data analytics.

Object Storage - S3 is unlike the others mentioned above. With the help of REST API or aws cli tool we can download or upload files easily. There is no format restrictions we can upload file of any kind.


Let's create our own EBS unit and use it.


  1. First need to create EC2 instance, click to the volumes section.






  2. And then create a volume, attach it to the EC2 instance we just created.







  3. Let's move on to the instance terminal and list all block storage available attached to this instance by lsblk  command. We will be able to see the block storage we just attached. it is 17 GB.


  4. Since this unit has no file system whatsoever our OS won't able to mount it, so we need to make file system by mkfs command as below.




  5. And then need to mount to the OS


  6. Let's create a file and save secret message into it.








  7. And detach the volume.




When we create a new EC2 instance and then attach volume to it, instead of making file-system again (If you use mkfs command again this will result to format that volume again eventually data loss) we just need to mount it directly to that instance. Finally we will able to see edit all the files, and of course our message file as well.



We can use EBS for other purposes as well, we can create AMI image snapshot and based on that create new EC2 instances easily.

  1.   Click to create image button.
     

  2. Name it and finish the operation. You will be able to see your AMI created here 



  3. Check out the snapshots, here is the EBS snapshot taken to create new image from it.



  4. All the  custom AMI's located in My AMI's section in the EC2 instance creation phase.


Consider that since the EBS snapshots are exact copy of the volume itself, this kind of mistakes might occur. If we create our EC2 instance based on the user-data I have provided to you and then create AMI from it. Previous EC2 instance's IP address will be shown in the index.html file. 


Let's create EFS storage unit and connect it to our EC2 instance

  1. Type EFS to the search bar and click to the icon as shown below





  2. Create EFS instance






  3. Just be sure the VPC you selected in previous step should match with the EC2 instance's subnet.







  4. Click to attach button






  5. You will be able to see command you need to connect via commandline


  6. Okay, make sure that you have already created the directory named "efs" as used in the command. When you navigate to that directory and create file there and connect to the same EFS instance using another EC2 instance you will see the files there too. This allows us to share files across different machines.





Comments

Post a Comment

Popular posts from this blog

S3 - Create a static website

Image
 S3 - Create a static website Creating static website with S3 bucket is very easy. This will make not only your website fast but also hassle-free and cheap alternative to traditional hosting. Please read this post before going further -  https://www.learn-aws.com/2024/11/s3.html  Create a bucket and set the permissions.  Click on the bucket, and navigate to the "Properties" tab, scroll down you will see this option. Edit it like below. Create a simple index.html file like the one below and upload it with the files you used in index file. In this case "aws.png" is also uploaded into the bucket as well. <html> <title> Hello from AWS S3 </title> <body> <img src="aws.png" alt="AWS"> </body> </html>  If we do everything right, we will be able to see rendered html when we copy and paste the link in the index file's link in S3 service.
Read more

AWS - Databases DynamoDB, RDS

Image
AWS - Databases There are 3 main database types in software world: Relational databases (MySql, Postgre, Amazon RDS etc.) Non-relational databases (Mongo, Amazon DynamoDB etc.) Graph databases (Amazon Neptune, NebulaGraph etc.) For your specific use case they have advantages and disadvantages.  We will cover relational and non-relational databases in this post.  Relational databases - as name implies stores information based on defined relation. Organized by tables, columns, rows. Supports complex queries and joins. Non-relational databases - contrary to the relational databases non-relational databases are more flexible. Documents, key/value, columns and graphs are some forms can be used to organize data. Databases in Amazon There are few ways to use database in AWS : Database on EC2 - We deploy any database software onto our instance and use it like this. Amazon RDS  - Amazon provided service, under the hood you can use (MySql,Postgre,Maria Amazon Aurora etc.) relationa...
Read more

S3

Image
 S3 In AWS S3 storage is special kind of storage that provides flexibility that other kind of storage services don't provide. S3 is sitting internet facing subnet and if we want to access S3 bucket from our private subnet we need to either direct traffic over internet gateway or use S3 gateway endpoint service.  To access or upload files to S3 we must use REST API for that. This actually makes things easier because thanks to this we will be able to do all operations programmatically (via our favorite programming language). In block storage service we supposed to use operating system to perform all the tasks. There should be some kind of hierarchy. But in S3 files storage in "bucket"s. Hierarchy can be mimicked by prefixes.  Let's create bucket and access over internet Search S3 in the search bar  Create a bucket Name must be unique otherwise it will display an error. Use default settings. After that try to upload some files there. Click to that file you will see the ...
Read more