google cloud platform – Big query Storage API permission issue
I am using a service account to write data to Big query using big query storage API using java SDK, and I see the below error. Do I need to provide any other roles to the service account?
Error: 00:44:38 | Error: com.google.api.gax.rpc.PermissionDeniedException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Request had insufficient authentication scopes.
Current Roles to the service account: BigQuery Admin, BigQuery Data Owner, BigQuery Data Editor
code snippet:
Map<String, Object> metadata = new HashMap<>();
metadata.put("table_name", "MetaData");
metadata.put("timestamp", Instant.now().toString());
metadata.put("is_processed", false);
JSONObject jsonObject = new JSONObject(metadata);
TableName parentTable = TableName.of("test", "test", "MetaData");
WriteStream stream = WriteStream.newBuilder().setType(WriteStream.Type.COMMITTED).build();
CreateWriteStreamRequest createWriteStreamRequest =
CreateWriteStreamRequest.newBuilder()
.setParent(parentTable.toString())
.setWriteStream(stream)
.build();
WriteStream writeStream = bigQueryWriteClient.createWriteStream(createWriteStreamRequest);
// Use the JSON stream writer to send records in JSON format.
// For more information about JsonStreamWriter, see:
// https://googleapis.dev/java/google-cloud-bigquerystorage/latest/com/google/cloud/bigquery/storage/v1/JsonStreamWriter.html
JsonStreamWriter streamWriter = JsonStreamWriter.newBuilder(writeStream.getName(), writeStream.getTableSchema()).build();
JSONArray arr = new JSONArray();
arr.put(jsonObject);
// Append asynchronously for increased throughput.
ApiFuture<AppendRowsResponse> future = streamWriter.append(arr, 1);
ApiFutures.addCallback(
future, new AppendCompleteCallback(), MoreExecutors.directExecutor());
Read more here: Source link