Available translations

API Map

foundations.png
api-map.png
An API Map represents the mapping of an API so that is can be consumed by bots within Superalgos. API Maps are created by users and shared as Plugins so that any other community member can use them with their bots.
The way to create an API Map is to read the API's documentation for each endpoint as well as the Data Mining Topic. These sources will provide you with all the information you need to make a successful API Map.
The basic way you can think about an API Map is as a representation of the URL link needed to request information from the API as well as the layout of the data returned by the API. This means that an API Map hold all the information needed to request data and decipher the data once it is returned.
How are API Maps used?
API Maps are used by API Data Fetcher Bots to request and save the data from an API. In order to do this a user needs to set up a task to run the bot. This set up is done automatically as part of the Install Market procedure if both an API Map and its data mine is detected within the Worksapce. From within that Task, a reference to the API Map is established.
When the Task is run, the UI will send to the Task Server all the definitions inside the API Map so that the API Data Fetcher Bot can use them to connect to the API Server, fetch data from it, and correctly interpret the responses.
How to Read an API Request URL
The API you are trying to map should have it's API Endpoints documented. If not, check any examples they may provide for how to request the data and see if there are inner folders below the main folder of the API Enpoint. Each one of these folders are an API Path Parameter that needs to be defined.
Here are some examples of API request URLs:
  https://api.coingecko.com/api/v3/coins/bitcoin?
 https://data.messari.io/api/v1/assets/btc?fields=id,slug,symbol,metrics/market_data
Sections of an API Request URL
Let's break down one of these URLs and identify each part of it.
  • Internet Protocol: This is the section at the beginning of almost any internet link you will see.
  https
  • Root Domain Name: This is the section of the URL that is before the first backslash:
  data.messari.io
  • API Path: This is the section that defines the main path to the API:
  api
  • API Version Number: This is the section that defines the API version to which you are making requests.
  v1
  • API Endpoint: This is the section that tells the API what kind of data you wish to request.
 assets
  • Path Parameters: These are the remaning sections of the path before the question mark they further narrow down what data you wish to request from the API. In this case we want to receive data about bitcoin:
  btc
  • Query Parameters: These are all the sections after the question mark. They are often used to pair down the amount of data returned by the API so that you only receive what you are interested in:
 fields=id,slug,symbol,metrics/market_data
Foundations->Topic->Data Mining - Fetching Data From APIs->Main Workflow
How to Fetch Data from an API?
The main workflow to pull this off is quite straight forward. It does not require coding at all. Follow this guide and you will be using data from any source on the Internet for your trading strategies in no time.
  • 1. Choose an API: The current version is limited to public APIs only. Secured APIs coming soon.
  • 2. Design an API Map: Read the APIs online documentation and identify the Endpoints that return the data you would like to fetch. Carefully map the Endpoints defining the parameters the API Server expects at each call, and also how the API Server response is formatted.
  • 3. Setup an API Data Fetcher Bot: Create a Data Mine for this API and add an API Data Fetcher Bot. From that Bot's Record Properties you will need to reference the fields returned by the API Server defined at the API Map. This is how you define which fields from each response you are going to save to disk. You might also need to define query or path parameters here.
  • 5. Setup Indicator Bots: Once the data is converted, you can extract specific Datasets from it. These datasets can be used to define a set of indicators you can build from that data. These Indicators can be as simple as reorganizations of the data (with some adjustments to the names of fields and splitting the data into different indicators), or it can include some data processing / transformations if you wish (this would require some coding).
  • 6. Setup Plotters: After you have set up your indicators you can design plotters for them if you wish.
  • 7. Use Data for Trading Strategies: These indicators can be consumed from your trading strategies.
Foundations->Topic->Data Mining - Fetching Data From APIs->Step By Step
Step by Step Guide to Fetch Data from an API
This is a detailed guide on how to setup a data mining operation fetching data from an Internet API.
  • Step #2: Open a second UI (this will prevent the previous one from saving the changes, but you don't need to do that). You will use the first UI to check from time to time how nodes are configured there, and the second UI to build your own data mining operation from scratch. The rest of the steps you will do within the second UI.
  • Step #3: Select an API from the Internet with data that might be useful at your Trading Strategy.
  • Step #4: Read the API's documentation, indentify the endpoints, what parameters they require and what data they will provide you. Another helpful way to see how the api responds is by typing the api endpoint into your browser and looking at the raw output.
  • Step #5: At Superalgos, get a clean workspace, without plugins pre-installed.
  • Step #6: Create an API Map node, and start mapping the documented API by adding children to it. Especially pay attention to unnamed arrays and objects that may wrap the data.
  • Step #7: For each node you add, read the node's doc page. This will save you time trying to guess how all this works.
  • Step #8: Define one API Endpoint first, once you have the first one working, go for the rest of the endpoints available.
  • Step #9: Define the query, or path parameters, and the responses. Read each node's help for details on how to do it right.
  • Step #11: Create a new Data Mine with the name of the company that hosts the API.
  • Step #13: Install a new market. This procedure is going to setup many things for you. It will detect the API Map, and the Data Mine you created and will create appropiate Tasks to run your API Fetcher Bot.
  • Step #14: Pay attention to the nodePath property at the API Response Schema node of the API Map. Check the node's docs page for details.
  • Step #17: The timestamp record property is the only required property. It helps keep the records you save organized. Additionally, the timestamp may live in the header of the API response. Read the docs of the API Response Field Reference node to set up the nodePath property if necessary.
  • Step #20: Configure the aggregationMethod property for each Record Property. Check the docs of that node for details.
  • Step #21: If you wish, you can create multiple Indicator Bots to cut the dataset in different groups of Record Properties.
  • Step #23: Now you are ready to run the Task where the Data Fetcher Bot Process Instance is defined.
  • Step #24: If something did not work, check the workspace map to see if there is a node with an error. Locate the node, click on the error and see the error's page at the doc. Try to understand what is wrong, fix it and try again.
  • Step #25: Iterate through step #24 until everything is working. Re read node's documentation until you get the details of what it explains there. Reread this guide or this whole Topic from the start, until the concepts explained sinks in.
  • Step #26: If after digesting the docs, and following all this advice you still cannot get it to work, go to the Telegram Support group and ask for help.
  • Step #27: Once all is running well, set up your Trading Strategy in a way that allows you to test if you have access to the processed data (the one provided by the latest Indicator Bots created).
  • Step #29: Once the data is correctly consumed by your Trading Strategy, iterate through this guide by adding another API Endpoint to the API Map. You will need to add an addtional API Fetcher Bot.
  • Step #30: Once you have successfully mapped the whole API and you have all the needed API Data Fetchers, Indicator Bots and Plotters tested, you are ready to share your API Map and Data Mine with the community as a plugin, and receive SA Tokens in exchange for your contribution.
API Map Configuration
Properties
  • codeName: this must be the same codename as the Data Mine codename in order for the Install New Market procedure to link them properly at the API Map Reference node.
  • protocol: the protocol to be used to access the API. For now either http or https.
  • portNumber: the port at which the API server is listening.
  • hostName: the domain name or IP address of the API server.
  • path: the URL path to where the API is located within the remote server.
  • millisecondsBetweenCalls: [Optional] the number of milliseconds to wait before calling the remote host. This is particuary useful if there is pagination, so that there is a delay before requesting each page.
Here is an example API Map config:
 {
    "codeName": "CoinGecko",
    "protocol": "https",
    "portNumber": "",
    "hostName": "api.coingecko.com",
    "path": "api",
    "millisecondsBetweenCalls": 4000
}
Initial Values
These are the Initial Values for API Map configuration:
 {
    "protocol": "https",
    "portNumber": "",
    "hostName": "",
    "path": ""
}
Examples
This is a list of properties featured by the API Map configuration. Expanding a property shows sample values for the property extracted from the current Workspace.
API Map Menu
The API Map node has the following Node Menu items:
The Configure menu item has the following properties:
  • action: Configure
  • actionFunction: uiObject.configEditor.activate
  • label: Configure
  • iconPathOn: configuration
  • iconPathOff: configuration
The Add API Version menu item has the following properties:
  • action: Add UI Object
  • label: Add API Version
  • actionFunction: payload.executeAction
  • actionProject: Visual-Scripting
  • relatedUiObjectProject: Foundations
The Delete menu item has the following properties:
  • action: Delete UI Object
  • actionProject: Visual-Scripting
  • askConfirmation: true
  • confirmationLabel: Confirm to Delete
  • label: Delete
  • iconPathOn: delete-entity
  • iconPathOff: delete-entity
  • actionFunction: payload.executeAction
API Map Children
The API Map node has the following childrenNodesProperties:
The Api Versions node property features the following properties:
  • name: apiVersions
  • type: array
  • autoAdd: true