google-cloud-storage

java.lang.NoClassDefFoundError error when I try to instantiate the GCS Service

主宰稳场 提交于 2020-02-05 04:05:07
问题 I am very new to java/Android development. My IDE of choice is Android Studio. Essentially I need to upload images/vidoes clips and audio clips to GCS. I am instantiating the service like this " final GcsService gcsService = >GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());" When I run the app I get the following error java.lang.NoClassDefFoundError: com.google.appengine.api.utils.SystemProperty at >com.google.appengine.tools.cloudstorage.GcsServiceFactory

Google Dataflow “No filesystem found for scheme gs”

走远了吗. 提交于 2020-02-04 05:54:05
问题 I'm trying to execute a Google Dataflow Application, but it is throw this Exception java.lang.IllegalArgumentException: No filesystem found for scheme gs at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(FileSystems.java:459) at org.apache.beam.sdk.io.FileSystems.matchNewResource(FileSystems.java:529) at org.apache.beam.sdk.io.FileBasedSink.convertToFileResourceIfPossible(FileBasedSink.java:213) at org.apache.beam.sdk.io.TextIO$TypedWrite.to(TextIO.java:700) at org.apache.beam.sdk

How to create a tsv file on Google Storage Transfer

安稳与你 提交于 2020-02-04 05:34:48
问题 Google provide a great documentation for their cloud services, but unfortunately nobody can understand the content. Their explanation always jumping and leave people with no clue to accomplish even a simple task. Creating tsv file should be a simple task. I tried to follow everything in this page Creating a URL List but stuck with "unknown error" log. Unknown error means I never know what I am doing wrong. They gave example downloaded file md5-test a string "Storage Transfer MD5 Test" =>

how to upload file line by line to google cloud storage in java

早过忘川 提交于 2020-02-04 01:20:25
问题 I have the following code: String fullFileUrl = fileUrl; Storage storage = StorageServiceHolder.getStorage(); BlobId blobId = GCSHelper.uri2blobId(fullFileUrl); while (!queue.isEmpty()) { try { String line = queue.poll(1000, TimeUnit.SECONDS); InputStream inputStream = new ByteArrayInputStream(line.getBytes()); BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build(); Blob blob = storage.create(blobInfo, inputStream); } catch (InterruptedException e) { log.error(

Stream BigQuery table into Google Pub/Sub

独自空忆成欢 提交于 2020-02-03 07:23:43
问题 I have a Google bigQuery Table and I want to stream the entire table into pub-sub Topic what should be the easy/fast way to do it? Thank you in advance, 回答1: That really depends on the size of the table. If it's a small table (a few thousand records, a couple doze columns) then you could setup a process to query the entire table, convert the response into a JSON array, and push to pub-sub. If it's a big table (millions/billions of records, hundreds of columns) you'd have to export to file,

Delete file in cloud storage from cloud functions

孤者浪人 提交于 2020-02-03 02:08:25
问题 I'm trying to make a Google Cloud Function which deletes images linked to person object in Firebase realtime database.. But every time I'm getting "Error during request" error (without any specific error.code, it's just undefined).. Here is a function code: const functions = require('firebase-functions'); // The Firebase Admin SDK to access the Firebase Realtime Database. const admin = require('firebase-admin'); const gcs = require('@google-cloud/storage')({ projectId: "PROJ_ID", keyFilename:

Google Storage access based on IP Address

陌路散爱 提交于 2020-02-01 16:10:28
问题 Is there a way to give access to a Google Cloud Storage bucket based on the IP address it is coming from. On Amazon s3, you can just set this in the access policy like this: "Condition" : { "IpAddress" : { "aws:SourceIp" : ["192.168.176.0/24","192.168.143.0/24"] } } I do not want to use a signed url. 回答1: UPDATE : This is now possible using VPC Service Controls No, this is not currently possible. There's currently a Feature request to restrict google cloud storage bucket by IP Address. 回答2:

GCS - Heap Error When File of higher size was uploaded

僤鯓⒐⒋嵵緔 提交于 2020-01-25 16:36:27
问题 I want to upload file size of 400MB to Google Cloud Storage. Following is the code snippet that I am using to get upload url and for posting in angularjs Java Servlet: UploadOptions opts = UploadOptions.Builder .withGoogleStorageBucketName(GCS_BUCKET_NAME); String url = blobstore.createUploadUrl("/upload", opts); AngularJS: $scope.formData.append('file', $scope.filename); $http.post(uploadUrl, $scope.formData, { withCredentials: true, headers: { 'Content-Type': undefined }, transformRequest:

GCS - Heap Error When File of higher size was uploaded

夙愿已清 提交于 2020-01-25 16:35:36
问题 I want to upload file size of 400MB to Google Cloud Storage. Following is the code snippet that I am using to get upload url and for posting in angularjs Java Servlet: UploadOptions opts = UploadOptions.Builder .withGoogleStorageBucketName(GCS_BUCKET_NAME); String url = blobstore.createUploadUrl("/upload", opts); AngularJS: $scope.formData.append('file', $scope.filename); $http.post(uploadUrl, $scope.formData, { withCredentials: true, headers: { 'Content-Type': undefined }, transformRequest:

Static Website Redirect HTTP to HTTPS with GCP Load Balancer with

大憨熊 提交于 2020-01-25 08:50:06
问题 I need to redirect my website from http to https , I have my static website in Google Cloud Storage Bucket pointed to Load Balance with http & https enabled. Example: http://ex.com = > https://ex.com http://www.ex.com = > https://www.ex.com The https://ex.com, https://www.ex.com both work just fine, however, I just need the http redirect so that I can reach Secure Service Since I am having a static website I hope I can only handle this in Load Balance, Can some one help me with this. 回答1: At