Node.js - Google Cloud Platform support
Introduction
Google Bigtable is a NoSQL database service in the cloud. This is the equivalent service to Amazon DynamoDB and Azure Cosmos DB offered by Google Cloud Platform (GCP). This extension, in the presence of the Bigtable API NodeJS library in source code, might create the following objects representing Bigtable tables.
Icon | Object | Name |
---|---|---|
NodeJS GCP Bigtable table | Name of the table (table id) | |
NodeJS GCP Unknown Bigtable table | Unknown |
A Bigtable table object will be created when the table ID (i.e. the table name) is resolved in calls to the method Table.table().
API method | Object creation |
---|---|
Table.table() | NodeJS GCP (Unknown) Bigtable table |
When the name of a given referred table in the code is not resolved, an unknown table object will be created with name Unknown. There can be a maximum of one unknown table per project. In addition to tables, links representing CRUD operations on Bigtable tables are created as well. The following methods are supported:
API methods (Table) | Link type |
---|---|
Table.createReadStream() | useSelectLink |
Table.deleteRows() | useDeleteLink |
Table.getRows() | useSelectLink |
Table.insert() | useInsertLink, useUpdateLink |
Table.mutate() | useUpdateLink, useInsertLink |
Table.sampleRowKeys() | useSelectLink |
Table.sampleRowKeysStream() | useSelectLink |
Table.truncate() | useDeleteLink |
API methods (Row) | Link type | Remarks |
---|---|---|
Row.create() | useInsertLink | Link created only when entries are passed in as options argument. |
Row.delete() | useDeleteLink | |
Row.deleteCells() | useUpdateLink | |
Row.filter() | useInsertLink, useUpdateLink | Link type presence will depend on input arguments. |
Row.get() | useSelectLink | |
Row.increment() | useInsertLink, useUpdateLink | |
Row.save() | useInsertLink, useUpdateLink |
The following code shows an example of TypeScript file querying data from a GCP Bigtable table via the Table.getRows() API method for NodeJS:
module.ts
import { Bigtable } from "@google-cloud/bigtable";
function my_get_books(){
const bigtable = new Bigtable();
const instanceId = 'instanceName';
const tableId = 'Books';
const instance = bigtable.instance(instanceId);
const table = instance.table(tableId);
const options = {
keys: ['title1', 'title2'],
};
table
.getRows(options)
.then(result => {
const rows = result[0];
})
.catch(err => {
// Handle the error.
});
}
The resulting Bigtable table object and the useSelectLink (Us):
Support for Google Cloud Storage
Google Cloud Storage is a file storage service in the cloud. This is the equivalent service to Amazon S3 provided by Google Cloud Platform (GCP). This extension, in the presence of the Nodejs client library for Google Storage in source code, might create the following objects representing Storage buckets:
Icon | Object | Name |
---|---|---|
NodeJS GCP Cloud Storage Bucket | Name of the bucket | |
NodeJS GCP Unknown Cloud Storage Bucket | Unknown |
We support the three relevant classes @google-cloud/storage for CRUD-like link creation: Storage, Bucket, and File. A Storage Bucket object will be created when its name is resolved in calls to the method Storage.bucket() for example*. *The two methods creating objects:
API method | Object creation |
---|---|
Storage.bucket() | NodeJS GCP (Unknown) Cloud Storage Bucket |
Storage.createBucket() | NodeJS GCP (Unknown) Cloud Storage Bucket |
When the name of a given referred bucket in the code is not resolved (either because of unavailable data or required technical complexity), an unknown bucket object will be created with name Unknown. There can be a maximum of one unknown table per project. Method calls involving CRUD-like operations are represented with links from the respective caller objects to Storage Buckets. The supported methods are listed in the table below.
CRUD-like API methods | Link type |
---|---|
Bucket.combine() | useInsertLink, useUpdateLink |
Bucket.deleteFiles() | useDeleteLink |
Bucket.upload() | useInsertLink, useUpdateLink |
File.copy() | useSelectLink, useInsertLink, useUpdateLink |
File.createReadStream() | useSelectLink |
File.createWriteStream() | useInsertLink, useUpdateLink |
File.delete() | useDeleteLink |
File.download() | useSelectLink |
File.save() | useInsertLink, useUpdateLink |
The following code illustrates a simple utilization of the NodeJS SDK for downloading a file from Google Cloud Storage:
downloadFile.ts
const bucketName = 'myBucketName';
import { Storage } from '@google-cloud/storage';
// Client
const storage = new Storage();
async function downloadFile() {
const options = {
destination: 'destFileName',
};
// Downloads the file
await storage.bucket(bucketName).file(fileName).download(options);
}
and the corresponding bucket object and useSelectLink created by the analyzer.
Limitations
- Files-list parameters from callbacks of Bucket.getFiles() and Bucket.getFilesStream() are not interpreted.
Support for Google Cloud Platform - PubSub
Google Cloud Platform Pub/Sub is a messaging service in the cloud that can be used to publish and subscribe to events.
Objects
This extension, when the Node.js client library for Google Pub/Sub is found in the source code, may create the following objects representing message publishers and message receivers:
Icon | Description |
---|---|
NodeJS GCP Pub/Sub Publisher | |
NodeJS GCP Unknown Pub/Sub Publisher | |
NodeJS GCP Pub/Sub Receiver | |
NodeJS GCP Unknown Pub/Sub Receiver |
- When a supported API for publishing to a Topic is found, this extension evaluates the Topic Name and creates a NodeJS GCP Pub/Sub Publisher object named as the topic.
- When a subscription API is found, this extension evaluates the subscription name as well as the name of the topic to which the subscription is made. This information is used by the com.castsoftware.wbslinker extension to link publishers with matching receivers.
- When a supported API for receiving messages from a subscription is found, this extension evaluates the subscription name and creates a NodeJS GCP Pub/Sub Receiver object named as the subscription.
API methods
Publishers
The supported “publish” API methods are:
API method | Object created | Link from caller | Remarks |
---|---|---|---|
Topic.publish | NodeJS GCP (Unknown) Pub/Sub Publisher | callLink | - |
Topic.publishJSON | NodeJS GCP (Unknown) Pub/Sub Publisher | callLink | - |
Topic.publishMessage | NodeJS GCP (Unknown) Pub/Sub Publisher | callLink | - |
Topic.resumePublishing | NodeJS GCP (Unknown) Pub/Sub Publisher | callLink | - |
PublisherClient.publish | NodeJS GCP (Unknown) Pub/Sub Publisher | callLink | - |
PublisherClient.topicPath | N/A | N/A | Method used to provide topics to the publication request. N/A: Not applicable. |
Pubsub.createTopic | N/A | N/A | Used for evaluation of publishing topics. |
PubSub.topic | N/A | N/A | Used for evaluation of publishing topics. |
Receivers
The supported “reception” API methods are:
API method | Object created | Link to handler (if resolved) from Receiver | Handler | Remarks |
---|---|---|---|---|
Subscription.on | NodeJS GCP (Unknown) Pub/Sub Receiver | callLink | Callback method passed as argument. | Only object created when first parameter is equal to ‘message’. |
SubscriberClient.pull | NodeJS GCP (Unknown) Pub/Sub Receiver | callLink | Parent of the object calling the API method | |
SubscriberClient.streamingPull | NodeJS GCP Unknown Pub/Sub Receiver | callLink | Parent of the object calling the API method | Limited support: only Unknown objects are created because of complexity to analyze this low-level API. |
SubscriberClient.subscriptionPath | N/A | N/A | N/A | Method used to provide subscription ids to the pull request. N/A: Not applicable. |
PublisherClient.topicPath | N/A | N/A | N/A | Method used to provide topics to the publication request. |
Pubsub.createTopic | N/A | N/A | N/A | Used for evaluation of subscription topics. |
PubSub.topic | N/A | N/A | N/A | Used for evaluation of subscription topics. |
Subscriptions
API method | Object created | Remarks |
---|---|---|
Topic.subscription | NodeJS GCP Pub/Sub Subscription | Contains a list of topics in “Topics subscribed to” property. |
Topic.createSubscription | NodeJS GCP Pub/Sub Subscription | As above |
PubSub.subscription | NodeJS GCP Pub/Sub Subscription | As above |
PubSub.createSubscription | NodeJS GCP Pub/Sub Subscription | As above |
Examples
Note
- In the following examples only the TypeScript extension is displayed. However, Publisher/Subscription/Receiver objects can be created by other languages (Java, DotNet, JavaScript).
- Reminder: the com.castsoftware.wbslinker extension is responsible for matching Publisher/Subscription/Receiver objects between different technologies according to the name/property matching protocol. Note that Unknown objects are not considered in this linking protocol.
Known/Unknown Publisher with Subscription and Receiver example
The TypeScript file named ’topic_publish_message.ts’ below publishes a message on topic my-topic:
topic_publish_message.ts
const {PubSub} = require('@google-cloud/pubsub');
const pubsub = new PubSub();
const topic = pubsub.topic('my-topic');
const data = Buffer.from('Hello, world!');
const callback = (err, messageId) => {
if (err) {
// Error handling omitted.
}
};
topic.publishMessage({data}, callback);
// unknown publisher
const topic2 = pubsub.topic(get_topic_name());
topic2.publishMessage({data}, callback);
In addition, the TypeScript file named ’topic_subscription_01.ts' creates a subscription to the same my-topic topic and the reception of the messages via the Subscription.on() API method:
topic_subscription_01.ts
const {PubSub} = require('@google-cloud/pubsub');
const pubsub = new PubSub();
const topic = pubsub.topic('my-topic');
const subscription = topic.subscription('my-subscription');
// Register a listener for `message` events.
subscription.on('message', (message) => {
});
This code will produce the following result:
The resulting objects created by the TypeScript extension are two Publishers (‘Unknown’ and ‘my-topic’) called by the TypeScript Module ’topic_publish_message.ts’ and one Receiver calling the anonymous function located in the second argument of the Subscription.on method call.
- Publishers/Receivers created by TypeScript reused Node.js metamodel objects.
- The Subscription ‘my-subscription’ is discovered by TypeScript extension but it is the Node.js extension which is responsible to create the object.
- The com.castsoftware.wbslinker extension is responsible for creating callLinks between Publishers/Receivers, matching Publisher/Subscription/Receiver objects between different technologies according to the name/property matching protocol. Note that Unknown objects are not considered in this linking protocol.
Multiple Publishing or Receiving example
The TypeScript file ‘publish.ts’ below publishes a message on topic based on the value of argument topicNameOrId in the async method ‘my_phys_caller’. This is an example of two different methods named ‘my_first_topic’ and ‘my_other_topic’ that declare a topic name and call method ‘my_phys_caller’ to create the topic and then publish the message:
publish.ts
const {PubSub} = require('@google-cloud/pubsub');
class PublisherCreation {
async my_phys_caller(topicNameOrId: String) {
// Instantiates a client
const pubsub = new PubSub();
// Creates a new topic
const topic = await pubsub.topic(topicNameOrId);
console.log(`Topic ${topic.name} created.`);
await topic.publish(Buffer.from('Test message!'));
}
my_first_topic() {
const topic_name = 'first-topic';
this.my_phys_caller(topic_name)
}
my_other_topic() {
const topic_name = 'other-topic';
this.my_phys_caller(topic_name)
}
}
Followed by the TypeScript file ‘receive_message.ts’ which listens to the subscription ‘my-sub’ attached to the topic ‘first-topic’ and ‘other-topic’ with the same logic as in the previous file:
subscription.ts
const {PubSub} = require('@google-cloud/pubsub');
class SubscriptionCreation {
async my_phys_caller(topicNameOrId: String) {
// Instantiates a client
const pubsub = new PubSub();
// Creates a new topic
const topic = await pubsub.topic(topicNameOrId);
const sub = topic.subscription('shared_sub');
// Register a listener for `message` events.
sub.on('message', (message) => {});
}
my_first_topic_attached_to_sub() {
const topic_name = 'first-topic';
this.my_phys_caller(topic_name);
}
my_other_topic_attached_to_sub() {
const topic_name = 'other-topic';
this.my_phys_caller(topic_name);
}
}
This code will produce the following result:
The resulting objects created by the TypeScript extension are two Publishers named ‘first-topic’ and ‘other-topic’ respectively called by method ‘my_first_topic’ and ‘my_other_topic’ and one Receiver calling the anonymous function located in the second argument of the Subscription.on method call. Typescript extension provide results about the Subscription named ‘my-sub’ and attached to topics ‘first-topic’ and ‘other-topic’ to Node.js extension.