knox-amazon-s3-client

nodejs knox put to s3 results in a 403

由 谁说我不能喝 提交于 2021-02-07 07:37:33
问题 I'm trying to upload to an Amazon s3 bucket using knox in a nodejs project but can't get past a 403 error. I've ensured that the key,secret, and bucket are properly set. I could really use some help here from those with more experience. My node code is as follows: var upload_test = function(){ var client = knox.createClient( { key: config.aws.key , secret: config.aws.secret , bucket: config.aws.bucket } ); fs.readFile('test.pdf', function(err,buf){ var req = client.put('6530/test.pdf', {

showing an image from amazon s3 with nodejs, expressjs and knox

由 余生长醉 提交于 2019-12-30 02:26:05
问题 I think this should be a straight forward thing, but I can't fing a solution :s I'm trying to figure out the best way to show images stored on amazon S3 on a website. Currently I'm trying to get this to work (unsuccessful) //app.js app.get('/test', function (req, res) { var file = fs.createWriteStream('slash-s3.jpg'); client.getFile('guitarists/cAtiPkr.jpg', function(err, res) { res.on('data', function(data) { file.write(data); }); res.on('end', function(chunk) { file.end(); }); }); }); /

How to make a socket a stream? To connect https response to S3 after imagemagick

由 好久不见. 提交于 2019-12-23 12:35:31
问题 I am node and programming in general, and I have been really struggling with this... I want to take a https response, resize it with graphicsmagick and send it to my amazon S3 bucket. It appears that the https res is an IncomingMessage object (I can't find any info about that) and the stdout from graphicsmagick is a Socket. The weird thing is that I can use pipe and send both of these to a writeStream with a local path, and both res and stdout create a nice new resized image. And I can even

Setting proxy server for connections in Knox

由 巧了我就是萌 提交于 2019-12-13 04:41:00
问题 I am using LearnBoost/knox to connect to Amazon S3 but my machine is using a proxy server. I have set an environment variable http_proxy and https_proxy but Knox is not using that. How do you get Knox to communicate through a proxy server. 回答1: A pull request for proxy support has been added to Knox about 8 months ago but it hasn't made it in the latest release yet (the pull request was closed). If you get that code, you can use: var client = knox.createClient({ key: '<api-key-here>' , secret

node.js & S3 / Write to s3 with knox

由 戏子无情 提交于 2019-12-13 02:18:00
问题 I'm trying to write to S3 with knox, by the following code: var knox = require('knox'); var client = knox.createClient({ key: 'key' , secret: 'pass' , bucket: S3_BUCKET }); fs.stat("/opt/files/" + url, function(err, stats) { if (stats != null && stats.size != 0){ var req = client.put(url, { 'Content-Length': stats.size }); req.on('error' ,function (err){ console.log(err); }) var readstr = fs.createReadStream("/opt/files/" + url); readstr.pipe(req); readstr.on('error', function (err){ console

Is Apache Knox Gateway compatible with Cloudera 4.5?

由 随声附和 提交于 2019-12-08 09:48:03
问题 I'm currently working on a future project with an Hadoop cluster. I need to find informations about security of the cluster. I found the API Apache Knox Gateway which seems to be what we need. We work with Cloudera 4.5 for now. In the future, we will upgrade to Cloudera 5. My problem is that Knox seems to not be compatible with Cloudera 4.5 (http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH-Version-and-Packaging-Information/cdhvd_topic_3.html). WebHDFS 2.4.0

Streaming file to S3 “Error: stream ended unexpectedly”

由 岁酱吖の 提交于 2019-12-08 02:34:35
问题 Update: I believe that this could be caused by the fact that I'm using the body parser provided by express. Could this be messing with the stream that multiparty is trying to parse? I'm basing my solution on this answer. What I'm trying to do: Stream a file from a client browser straight to S3 with my NodeJS server acting as a proxy (for security purposes). I don't want the file to touch the server's filesystem to avoid that bottleneck. I'm getting the following error: events.js:72 throw er;

node application stream file upload directly to amazon s3 [duplicate]

由 我的未来我决定 提交于 2019-12-06 12:03:59
问题 This question already has answers here : How to upload a file from the browser to Amazon S3 with node.js, Express, and knox? [closed] (6 answers) Closed 6 years ago . Hello :) I'm looking at the feasibility of having my node application stream HTTP POST file uploads directly through to an Amazon S3 Bucket. I'm looking at using Formidable and Knox for this. The part I'm unsure about is that S3 requires that you know the total number of bytes in the file before transmission. Would I be right in

Streaming file to S3 “Error: stream ended unexpectedly”

由 风流意气都作罢 提交于 2019-12-06 11:36:29
Update: I believe that this could be caused by the fact that I'm using the body parser provided by express. Could this be messing with the stream that multiparty is trying to parse? I'm basing my solution on this answer . What I'm trying to do: Stream a file from a client browser straight to S3 with my NodeJS server acting as a proxy (for security purposes). I don't want the file to touch the server's filesystem to avoid that bottleneck. I'm getting the following error: events.js:72 throw er; // Unhandled 'error' event ^ Error: stream ended unexpectedly at Form.<anonymous> (/Users/kentcdodds

node application stream file upload directly to amazon s3 [duplicate]

由 末鹿安然 提交于 2019-12-04 18:35:35
This question already has answers here : How to upload a file from the browser to Amazon S3 with node.js, Express, and knox? [closed] (6 answers) Closed 6 years ago . Hello :) I'm looking at the feasibility of having my node application stream HTTP POST file uploads directly through to an Amazon S3 Bucket. I'm looking at using Formidable and Knox for this. The part I'm unsure about is that S3 requires that you know the total number of bytes in the file before transmission. Would I be right in thinking that the only way to accomplish this then would be to use the HTML5 File API (And possibly an
订阅 knox-amazon-s3-client