Node.js - Google Cloud Platform support

Introduction

Google Bigtable is a NoSQL database service in the cloud. This is the equivalent service to Amazon DynamoDB and Azure Cosmos DB offered by Google Cloud Platform (GCP). This extension, in the presence of the Bigtable API NodeJS library in source code, might create the following objects representing Bigtable tables.

Icon Object Name
NodeJS GCP Bigtable table Name of the table (table id)
NodeJS GCP Unknown Bigtable table Unknown

A Bigtable table object will be created when the table ID (i.e. the table name) is resolved in calls to the method Table.table().

API method Object creation
Table.table() NodeJS GCP (Unknown) Bigtable table

When the name of a given referred table in the code is not resolved, an unknown table object will be created with name Unknown. There can be a maximum of one unknown table per project. In addition to tables, links representing CRUD operations on Bigtable tables are created as well. The following methods are supported:

API methods (Table) Link type
Table.createReadStream() useSelectLink
Table.deleteRows() useDeleteLink
Table.getRows() useSelectLink
Table.insert() useInsertLink, useUpdateLink
Table.mutate() useUpdateLink, useInsertLink
Table.sampleRowKeys() useSelectLink
Table.sampleRowKeysStream() useSelectLink
Table.truncate() useDeleteLink
API methods (Row) Link type Remarks
Row.create() useInsertLink Link created only when entries are passed in as options argument.
Row.delete() useDeleteLink
Row.deleteCells() useUpdateLink
Row.filter() useInsertLink, useUpdateLink Link type presence will depend on input arguments.
Row.get() useSelectLink
Row.increment() useInsertLink, useUpdateLink
Row.save() useInsertLink, useUpdateLink

The following code shows an example of TypeScript file querying data from a GCP Bigtable table via the Table.getRows() API method for NodeJS:

module.ts

import { Bigtable } from "@google-cloud/bigtable";

function my_get_books(){
    const bigtable = new Bigtable();
     
    const instanceId = 'instanceName';
    const tableId = 'Books';
    const instance = bigtable.instance(instanceId);
    const table = instance.table(tableId);
     
    const options = {
      keys: ['title1', 'title2'],
    };
    table
      .getRows(options)
      .then(result => {
        const rows = result[0];
      })
      .catch(err => {
        // Handle the error.
      });
}

The resulting Bigtable table object and the useSelectLink (Us):

Alt text

Support for Google Cloud Storage

Google Cloud Storage is a file storage service in the cloud. This is the equivalent service to Amazon S3 provided by Google Cloud Platform (GCP). This extension, in the presence of the Nodejs client library for Google Storage in source code, might create the following objects representing Storage buckets:

Icon Object Name
NodeJS GCP Cloud Storage Bucket Name of the bucket
NodeJS GCP Unknown Cloud Storage Bucket Unknown

We support the three relevant classes @google-cloud/storage for CRUD-like link creation: Storage, Bucket, and File. A Storage Bucket object will be created when its name is resolved in calls to the method Storage.bucket() for example*. *The two methods creating objects:

API method Object creation
Storage.bucket() NodeJS GCP (Unknown) Cloud Storage Bucket
Storage.createBucket() NodeJS GCP (Unknown) Cloud Storage Bucket

When the name of a given referred bucket in the code is not resolved (either because of unavailable data or required technical complexity), an unknown bucket object will be created with name Unknown. There can be a maximum of one unknown table per project. Method calls involving CRUD-like operations are represented with links from the respective caller objects to Storage Buckets. The supported methods are listed in the table below.

CRUD-like API methods Link type
Bucket.combine() useInsertLink, useUpdateLink
Bucket.deleteFiles() useDeleteLink
Bucket.upload() useInsertLink, useUpdateLink
File.copy() useSelectLink, useInsertLink, useUpdateLink
File.createReadStream() useSelectLink
File.createWriteStream() useInsertLink, useUpdateLink
File.delete() useDeleteLink
File.download() useSelectLink
File.save() useInsertLink, useUpdateLink

The following code illustrates a simple utilization of the NodeJS SDK for downloading a file from Google Cloud Storage:

downloadFile.ts

const bucketName = 'myBucketName';
import { Storage } from '@google-cloud/storage';

// Client
const storage = new Storage();

async function downloadFile() {
    
    const options = {
      destination: 'destFileName',
    };

    // Downloads the file
    await storage.bucket(bucketName).file(fileName).download(options);
}

and the corresponding bucket object and useSelectLink created by the analyzer.

Limitations

  • Files-list parameters from callbacks of Bucket.getFiles() and Bucket.getFilesStream() are not interpreted.

Support for Google Cloud Platform - PubSub

Google Cloud Platform Pub/Sub is a messaging service in the cloud that can be used to publish and subscribe to events.

Objects

This extension, when the Node.js client library for Google Pub/Sub is found in the source code, may create the following objects representing message publishers, topic subscriptions and message receivers:

Icon Description
NodeJS GCP Pub/Sub Publisher
NodeJS GCP Unknown Pub/Sub Publisher
NodeJS GCP Pub/Sub Subscription
NodeJS GCP Pub/Sub Receiver
NodeJS GCP Unknown Pub/Sub Receiver
  • When a supported API for publishing to a Topic is found, this extension evaluates the Topic Name and creates a NodeJS GCP Pub/Sub Publisher object named as the topic.
  • When a subscription API is found, this extension evaluates the subscription name as well as the name of the topic to which the subscription is made and creates a NodeJS GCP Pub/Sub Subscription object and saves the name of the topic in its “Topics subscribed to” property.
  • When a supported API for receiving messages from a subscription is found, this extension evaluates the subscription name and creates a NodeJS GCP Pub/Sub Receiver object named as the subscription.
  • For publisher or receiver, if the evaluation of the topic name or subscription name fails an Unknown publisher or Unknown receiver object named Unknown is created.

The com.castsoftware.wbslinker extension* *is responsible for matching Publisher/Subscription/Receiver objects between different technologies according to the name/property matching protocol. Note that Unknown objects are not considered in this linking protocol.

API methods

Publishers

The supported “publish” API methods are:

API method Object created Link from caller Remarks
Topic.publish NodeJS GCP (Unknown) Pub/Sub Publisher callLink -
Topic.publishJSON NodeJS GCP (Unknown) Pub/Sub Publisher callLink -
Topic.publishMessage NodeJS GCP (Unknown) Pub/Sub Publisher callLink -
Topic.resumePublishing NodeJS GCP (Unknown) Pub/Sub Publisher callLink -
PublisherClient.publish NodeJS GCP (Unknown) Pub/Sub Publisher callLink -
PublisherClient.topicPath N/A N/A Method used to provide topics to the publication request. N/A: Not applicable.
Pubsub.createTopic N/A N/A Used for evaluation of publishing topics.
PubSub.topic N/A N/A Used for evaluation of publishing topics.

Receivers

The supported “reception” API methods are:

API method Object created Link to handler (if resolved) from Receiver Handler Remarks
Subscription.on NodeJS GCP (Unknown) Pub/Sub Receiver callLink Callback method passed as argument. Only object created when first parameter is equal to ‘message’.
SubscriberClient.pull NodeJS GCP (Unknown) Pub/Sub Receiver callLink Parent of the object calling the API method
SubscriberClient.streamingPull NodeJS GCP Unknown Pub/Sub Receiver callLink Parent of the object calling the API method Limited support: only Unknown objects are created because of complexity to analyze this low-level API.
SubscriberClient.subscriptionPath N/A N/A N/A Method used to provide subscription ids to the pull request. N/A: Not applicable.
PublisherClient.topicPath N/A N/A N/A Method used to provide topics to the publication request.
Pubsub.createTopic N/A N/A N/A Used for evaluation of subscription topics.
PubSub.topic N/A N/A N/A Used for evaluation of subscription topics.

Subscriptions

API method Object created Remarks
Topic.subscription NodeJS GCP Pub/Sub Subscription Contains a list of topics in “Topics subscribed to” property.
Topic.createSubscription NodeJS GCP Pub/Sub Subscription As above
PubSub.subscription NodeJS GCP Pub/Sub Subscription As above
PubSub.createSubscription NodeJS GCP Pub/Sub Subscription As above

Examples

Note

  • In the following examples only the TypeScript extension is displayed. However, Publisher/Subscription/Receiver objects can be created by other languages (Java, DotNet, JavaScript).
  • Reminder: the com.castsoftware.wbslinker  extension is responsible for matching Publisher/Subscription/Receiver objects between different technologies according to the name/property matching protocol. Note that Unknown objects are not considered in this linking protocol.  

Known/Unknown Publisher with Subscription and Receiver example

The TypeScript file named ’topic_publish_message.ts’ below publishes a message on topic my-topic:

topic_publish_message.ts

const {PubSub} = require('@google-cloud/pubsub');
const pubsub = new PubSub();
const topic = pubsub.topic('my-topic');

const data = Buffer.from('Hello, world!');

const callback = (err, messageId) => {
  if (err) {
    // Error handling omitted.
  }
};

topic.publishMessage({data}, callback);


// unknown publisher
const topic2 = pubsub.topic(get_topic_name());
topic2.publishMessage({data}, callback);

In addition, the TypeScript file named ’topic_subscription_01.ts' creates a subscription to the same my-topic topic and the reception of the messages via the Subscription.on() API method:

topic_subscription_01.ts

const {PubSub} = require('@google-cloud/pubsub');
const pubsub = new PubSub();

const topic = pubsub.topic('my-topic');
const subscription = topic.subscription('my-subscription');

// Register a listener for `message` events.
subscription.on('message', (message) => {
});  

This code will produce the following result:

Alt text

The resulting objects created by the TypeScript extension are two Publishers (‘Unknown’ and ‘my-topic’) called by the TypeScript module ’topic_publish_message.ts’ and one Receiver calling the anonymous function located in the second argument of the Subscription.on method call. 

  • Publishers/Receivers created by TypeScript reused Node.js metamodel objects.
  • The Subscription ‘my-subscription’ is discovered by TypeScript extension but it is the Node.js extension which is responsible to create the object.
  • The com.castsoftware.wbslinker extension is responsible for creating callLinks between Publishers/Subscription and Subscription/Receivers.

Multiple Publishing or Receiving example

The TypeScript file ‘publish.ts’ below publishes a message on topic based on the value of argument topicNameOrId in the async method ‘my_phys_caller’. This is an example of two different methods named ‘my_first_topic’  and ‘my_other_topic’ that declare a topic name and call method ‘my_phys_caller’ to create the topic and then publish the message:

publish.ts

const {PubSub} = require('@google-cloud/pubsub');
class PublisherCreation {
    async my_phys_caller(topicNameOrId: String) {
        // Instantiates a client
        const pubsub = new PubSub();
        // Creates a new topic
        const topic = await pubsub.topic(topicNameOrId);
        console.log(`Topic ${topic.name} created.`);
        await topic.publish(Buffer.from('Test message!'));
    }

    my_first_topic() {
        const topic_name = 'first-topic';
        this.my_phys_caller(topic_name)
    }

    my_other_topic() {
        const topic_name = 'other-topic';
        this.my_phys_caller(topic_name)
    }
}

Followed by the TypeScript file ‘receive_message.ts’ which listens to the subscription ‘my-sub’ attached to the topic ‘first-topic’ and ‘other-topic’ with the same logic as in the previous file:

receive_message.ts

const {PubSub} = require('@google-cloud/pubsub');
class SubscriptionCreation {
    
    async my_phys_caller(topicNameOrId: String) {
        // Instantiates a client
        const pubsub = new PubSub();
        // Creates a new topic
        const topic = await pubsub.topic(topicNameOrId);
        const sub = topic.subscription('my-sub');
        // Register a listener for `message` events.
        sub.on('message', (message) => {});  
    }

    my_first_topic_attached_to_sub() {
        const topic_name = 'first-topic';
        this.my_phys_caller(topic_name);
    }

    my_other_topic_attached_to_sub() {
        const topic_name = 'other-topic';
        this.my_phys_caller(topic_name);
    }
}

This code will produce the following result:

The resulting objects created by the TypeScript extension are two Publishers named ‘first-topic’ and ‘other-topic’ respectively called by method ‘my_first_topic’ and ‘my_other_topic’ and one Receiver calling the anonymous function located in the second argument of the Subscription.on method call. Typescript extension provide results about the Subscription named ‘my-sub’ and attached to topics ‘first-topic’ and ‘other-topic’ to Node.js extension.