Re: Possible to bring in data stored on S3?

From: Jennifer Staab <no email>
Date: Mon, 06 Oct 2014 21:49:54 -0400

I agree with Nik, it seems reasonably fast in my experience. Note that
S3 doesn't have a file system, it's just an indexed object. So you can
write your object once and read it many times, but there isn't any
updating, file locking, etc. I typically use S3 for storage and
archiving or to provide a static data reference for things that aren't
going to change, but need to be widely available. I know those that use
S3 for static webpages, or for storing logs, etc. If your data is going
to change often, you might want to do things a different way.

Good Luck.

Jennifer

On 10/6/14 9:19 PM, John Readey wrote:
> Is anyone pulling in data on demand using the S3 API's? I'm curious
> how the performance compares to using NFS.
>
> John
>
> From: Jennifer Staab <jstaab_at_cs.unc.edu <mailto:jstaab_at_cs.unc.edu>>
> Date: Monday, October 6, 2014 at 6:08 PM
> To: greg <margeemail_at_gmail.com <mailto:margeemail_at_gmail.com>>,
> "starcluster_at_mit.edu <mailto:starcluster_at_mit.edu>"
> <starcluster_at_mit.edu <mailto:starcluster_at_mit.edu>>
> Subject: Re: [StarCluster] Possible to bring in data stored on S3?
>
> Besides the others mentioned, AWS has their own "command line
> interface" CLI see here
> <http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html>
> for installation instructions and here
> <http://docs.aws.amazon.com/cli/latest/> for usage. AWS CLI
> provides tools for accessing S3. If you don't want to use AWS CLI,
> there are many SDKs available depending on programming language or
> platform see here <http://aws.amazon.com/tools/>.
>
> Good Luck.
>
> Jennifer
>
> On 10/6/14 3:29 PM, greg wrote:
>> Hi all,
>>
>> Is it possible to bring data into my cluster from Amazon S3? I didn't
>> see it in the manual.
>>
>> Thanks for reading!
>>
>> Greg
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.eduhttp://mailman.mit.edu/mailman/listinfo/starcluster
>
Received on Mon Oct 06 2014 - 21:49:58 EDT