Day 41: S3 Programmatic access with AWS-CLI πŸ’» πŸ“

Day 41: S3 Programmatic access with AWS-CLI πŸ’» πŸ“

Β·

10 min read

πŸš€ How to Enable S3 Programmatic Access with AWS CLI 🌟

Introduction: So, you're ready to take control of your AWS S3 buckets programmatically using the AWS CLI! With a few simple steps, you can unleash the power of automation and manage your S3 resources efficiently. Let's dive in and get started!

Step 1: πŸ› οΈ Install AWS CLI If you haven't already, install the AWS CLI on your system. You can do this using pip (Python package installer) with the following command:

Copy codepip install awscli

Step 2: πŸ”‘ Configure AWS CLI Once installed, configure the AWS CLI with your AWS credentials. Open a terminal and type:

Copy codeaws configure

You will be prompted to enter your AWS Access Key ID, Secret Access Key, default region, and default output format.

Step 3: πŸšͺ Create IAM User To grant programmatic access to your S3 buckets, you need to create an IAM user with appropriate permissions. Navigate to the IAM console in your AWS Management Console and follow these steps:

  1. Click on "Users" and then "Add user".

  2. Enter a username and select "Programmatic access" as the access type.

  3. Attach policies that grant necessary S3 permissions to the user. For example, you can attach the "AmazonS3FullAccess" policy for full access to S3.

Step 4: πŸ—οΈ Access Key and Secret After creating the user, you'll be provided with an Access Key ID and a Secret Access Key. Make sure to copy these as you'll need them to configure the AWS CLI.

Step 5: πŸ›‘οΈ Secure Access Keys Store your Access Key ID and Secret Access Key securely. Never expose them in your code or share them publicly.

Step 6: πŸ”„ Test Configuration To verify that your AWS CLI is properly configured, run a simple command like listing S3 buckets:

bashCopy codeaws s3 ls

If configured correctly, you should see a list of your S3 buckets.

Step 7: πŸŽ‰ You're Ready to Go! Congratulations! πŸ₯³ You now have programmatic access to your AWS S3 buckets using the AWS CLI. You can start performing various operations like uploading, downloading, and managing objects in your buckets right from your command line interface.

πŸ”„S3

Amazon Simple Storage Service (S3) πŸ“¦πŸš€

Amazon S3 is like a giant virtual storage warehouse 🏬 in the cloud. It's a highly scalable, durable, and secure object storage service offered by Amazon Web Services (AWS). Here's what each emoji represents:

πŸ“¦ Objects: At the heart of S3 are objects, which can be anything from documents πŸ“„, images πŸ–ΌοΈ, videos πŸ“Ή, to any other type of file. Each object is stored in a bucket.

🏬 Buckets: Buckets are like containers πŸ›’οΈ where you store your objects. Think of them as folders, but in the cloud. You can create multiple buckets to organize your data logically.

πŸ” Security: S3 provides robust security features to protect your data. You can control access to your buckets and objects using access control lists (ACLs), bucket policies, and IAM policies. This ensures that only authorized users can access your data.

πŸ’» Durability: S3 is designed for 99.999999999% (11 nines) durability of objects over a given year. Your data is replicated across multiple devices and facilities within a region to ensure high durability and availability.

πŸš€ Scalability: Whether you have a few gigabytes or petabytes of data, S3 can scale to meet your storage needs. You can store virtually unlimited amounts of data in S3 without worrying about infrastructure management.

🌐 Accessibility: S3 is accessible over the internet, allowing you to store and retrieve data from anywhere in the world. This makes it ideal for hosting static websites, storing backups, and serving as a content delivery network (CDN) origin.

πŸ”„ Versioning: S3 supports versioning, which means you can keep multiple versions of an object in the same bucket. This helps protect against accidental deletion or overwriting of objects.

πŸ”— Integration: S3 integrates seamlessly with other AWS services like Lambda, EC2, Glacier, and many more. This enables you to build powerful, scalable applications using S3 as a central storage component.

In summary, Amazon S3 is a versatile, reliable, and scalable storage service that allows you to store and retrieve any amount of data securely in the cloud. Whether you're a small startup or a large enterprise, S3 provides the foundation for building robust and scalable applications in the AWS ecosystem.

πŸ”—TASK 01

Launch an EC2 instance using the AWS Management Console and connect to it using Secure Shell (SSH).

Sure, here's a step-by-step guide to launching an EC2 instance using the AWS Management Console and connecting to it using Secure Shell (SSH):

  1. Sign in to the AWS Management Console: Open your web browser and go to the AWS Management Console (console.aws.amazon.com). Sign in with your AWS account credentials.

  2. Navigate to EC2 Dashboard: Once logged in, navigate to the EC2 service by clicking on "Services" in the top left corner and selecting "EC2" under the Compute section.

  3. Launch Instance: On the EC2 dashboard, click on the "Launch Instance" button to start the instance creation process.

  4. Choose an Amazon Machine Image (AMI): Select an AMI from the list provided. You can choose an Amazon Linux, Ubuntu, or any other suitable AMI depending on your requirements. Click on the "Select" button.

  5. Choose an Instance Type: Choose an instance type based on your workload requirements. For this example, you can select a t2.micro instance type, which is included in the AWS Free Tier. Click on the "Next: Configure Instance Details" button.

  6. Configure Instance Details: Optionally, configure instance details such as network settings, IAM role, monitoring, etc. For beginners, you can leave the default settings. Click on the "Next: Add Storage" button.

  7. Add Storage: Specify the size of the root volume for your instance. The default size should be sufficient for most cases. Click on the "Next: Add Tags" button.

  8. Add Tags (Optional): Optionally, add tags to your instance for better organization and management. Tags are key-value pairs that help you identify your resources. Click on the "Next: Configure Security Group" button.

  9. Configure Security Group: Configure the security group settings to control the traffic to your instance. Ensure that you allow SSH (port 22) access from your IP address or a specific range of IP addresses. Click on the "Review and Launch" button.

  10. Review and Launch: Review the instance configuration to ensure everything looks correct. If satisfied, click on the "Launch" button.

  11. Create a New Key Pair or Choose an Existing One: Select an existing key pair or create a new one. This key pair is used for SSH authentication. Once selected or created, acknowledge and click on the "Launch Instances" button.

  12. Launch Status: Wait for the instance to be launched. Once it's launched, you'll see a confirmation message. Click on the "View Instances" button.

  13. Retrieve Public IP Address: In the EC2 dashboard, find your newly launched instance. Note down its public IP address. You'll need this to connect to the instance via SSH.

  14. Connect to EC2 Instance using SSH: Open your terminal (on macOS/Linux) or Command Prompt (on Windows). Use the following command to connect to your EC2 instance:

    cssCopy codessh -i /path/to/your/keypair.pem ec2-user@your-public-ip
    

    Replace /path/to/your/keypair.pem with the path to your private key file and your-public-ip with the public IP address of your EC2 instance.

  15. Authenticate SSH Connection: When prompted, type "yes" to authenticate the SSH connection. If successful, you'll be logged in to your EC2 instance via SSH.

Congratulations! πŸŽ‰ You've successfully launched an EC2 instance using the AWS Management Console and connected to it using Secure Shell (SSH). Now you can start using your EC2 instance for your desired applications and workloads.

πŸ”— TASK 02

Create an S3 bucket and upload a file to it using the AWS Management Console.

Sure, here's a step-by-step guide to creating an S3 bucket and uploading a file to it using the AWS Management Console:

  1. Sign in to the AWS Management Console: Open your web browser and navigate to the AWS Management Console (console.aws.amazon.com). Sign in with your AWS account credentials.

  2. Navigate to S3 Dashboard: Once logged in, navigate to the S3 service by clicking on "Services" in the top left corner and selecting "S3" under the Storage section.

  3. Create a New Bucket: On the S3 dashboard, click on the "Create bucket" button to create a new bucket.

  4. Configure Bucket Settings: Enter a unique name for your bucket. Bucket names must be globally unique across all of AWS, so choose a name that is not already in use. Select the region for your bucket. You can also configure additional settings such as versioning, server access logging, and tags if needed. Once done, click on the "Create bucket" button.

  5. Bucket Created: Your bucket has been successfully created. You'll be redirected to the bucket dashboard.

  6. Upload a File: To upload a file to the bucket, click on the name of the bucket you just created. Then click on the "Upload" button.

  7. Select File: Click on the "Add files" button to select the file you want to upload from your local machine. You can also drag and drop files into the upload area.

  8. Configure Upload Options: Optionally, you can configure additional settings for the upload such as encryption, access permissions, storage class, etc. Leave these as default if you're unsure. Click on the "Upload" button to start the upload process.

  9. File Uploaded: Once the upload is complete, you'll see a confirmation message indicating that the file has been successfully uploaded to your S3 bucket.

  10. Access the Uploaded File: You can now access the uploaded file by clicking on its name in the bucket dashboard. You'll see details about the file such as its size, storage class, and permissions.

Congratulations! πŸŽ‰ You've successfully created an S3 bucket and uploaded a file to it using the AWS Management Console. Your file is now securely stored in Amazon S3 and can be accessed and managed as needed.

Access the file from the EC2 instance using the AWS Command Line Interface (AWS CLI).

To access the file from the EC2 instance using the AWS Command Line Interface (AWS CLI), you'll first need to ensure that the EC2 instance has the necessary permissions to access the S3 bucket where the file is stored. You can do this by attaching an IAM role to the EC2 instance with permissions to access the S3 bucket.

Once the EC2 instance has the required permissions, you can use the AWS CLI installed on the instance to download the file from the S3 bucket. Here are the steps:

  1. Attach IAM Role to EC2 Instance (If not already done): If you haven't already attached an IAM role to the EC2 instance with permissions to access the S3 bucket, you can do so by following these steps:

    • Go to the EC2 dashboard in the AWS Management Console.

    • Select your EC2 instance.

    • In the details pane at the bottom, under the "Description" tab, locate the "IAM role" field and click on the associated IAM role link.

    • In the IAM console, attach a policy that grants read access to the S3 bucket containing the file.

  2. SSH into the EC2 Instance: Use SSH to connect to your EC2 instance. Open your terminal (on macOS/Linux) or Command Prompt (on Windows) and use the following command:

     cssCopy codessh -i /path/to/your/keypair.pem ec2-user@your-ec2-public-ip
    

    Replace /path/to/your/keypair.pem with the path to your private key file and your-ec2-public-ip with the public IP address of your EC2 instance.

  3. Use AWS CLI to Download the File: Once logged in to the EC2 instance, you can use the AWS CLI to download the file from the S3 bucket. Use the aws s3 cp command followed by the S3 URI of the file and the local destination where you want to save the file. For example:

     rubyCopy codeaws s3 cp s3://your-bucket-name/path/to/your/file /path/to/local/destination
    

    Replace your-bucket-name with the name of your S3 bucket, path/to/your/file with the path to the file in the bucket, and /path/to/local/destination with the local directory where you want to save the file.

  4. Verify File Download: After executing the command, the file will be downloaded to the specified local destination on your EC2 instance. You can verify the download by checking the specified directory.

That's it! You have now accessed the file from the EC2 instance using the AWS CLI.

Β