[talk] S3 Work-a-likes

[Edward Capriolo]

On Sun, Nov 13, 2016 at 6:58 AM, Sujit K M <kmsujit at gmail.com> wrote:

> > Is the language only Java, I am quite unaware of Python. Totally
> > Unaware of S3 Too.
> As a JAVA Developer I found a lot of issues in what seems the so called
> "implementing “object stores” for immutable data collections", but as was
> surprising to me didn't find it in other languages like Python.
>
> I got some documentton on Python on the Concurrent Access Violations.
> https://docs.python.org/2/library/exceptions.html (Surprisingly seems
> to be only RuntimeError)
> https://docs.python.org/3.1/library/pickle.html (has a line on
> concurrency control)
>
> Below some documentation on Java.
> http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> http://lists.nycbug.org/mailman/listinfo/talk


Typically I would not consider S3 an object store, (DynamoDB is more the
type of system I would suggest for storing data collections)

S3 is designed to store large objects (files) in particular the only way to
search it is by paging the files though an api.

http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html

public ObjectListing
<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ObjectListing.html>
listObjects(String
<http://docs.oracle.com/javase/6/docs/api/java/lang/String.html?is-external=true>
bucketName,
                                 String
<http://docs.oracle.com/javase/6/docs/api/java/lang/String.html?is-external=true>
prefix)
                          throws SdkClientException
<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/SdkClientException.html>,
                                 AmazonServiceException
<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/AmazonServiceException.html>

Most of the amazon API's have small max page size. (100,1000). In
cases where you do not know the name and you have many files you end
up paging and each page is an API call.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nycbug.org/pipermail/talk/attachments/20161113/b0d6a83d/attachment.html>