The Cloud Conformity VPC Endpoint Exposed check brought my attention to this one, as I think they have a bug in this check: https://www.cloudconformity.com/knowledge-base/aws/VPC/endpoint-exposed.html There are two types of VPC endpoint: Interface and Gateway (S3, DynamoDB). What are suggested in the above link only applies to the Interface endpoint, but not the Gateway endpoint. As … Continue reading Endpoint policies for gateway endpoints
Tag: S3
`aws s3 sync` lessons learned
As mentioned in my earlier posts, we use aws s3 sync to migrate a large number of files from on-premise to AWS. Here are a few things that I learned: The AWS credential can be renewed by an external process for a long running AWS CLI process. (Renew AWS credential for a long run AWS … Continue reading `aws s3 sync` lessons learned
AWS DataSync vs S3 Sync
I am currently working on a data migration project (from on-premise to AWS cloud). Also I would like to use the method to sync the data from AWS back to on-premise for DR purpose after we have done the migration. The total data size is about 1TB, and it is an online application data which … Continue reading AWS DataSync vs S3 Sync
Cross-Region S3 Replication Monitor
Reference: https://aws.amazon.com/answers/infrastructure-management/crr-monitor/
Get S3 Bucket Size
I did not realize that you can not find out the bucket size in the AWS console until some one asked me today. Here is a quick command to get the total size. % for each in bucket1 bucket2; do echo -n $each; aws s3 ls --summarize --human-readable --recursive s3://$each | grep -i 'Total Size'; … Continue reading Get S3 Bucket Size
Jackie Chen's IT Workshop 