Datasets

We can consume the metadata of dataset-type resources published in an Open Data Portal using a GET request. These GET requests can return either a list of all published dataset resources, or a single GUID-identified resource.

GET   /api/v2/datasets.json
GET   /api/v2/datasets/{guid}.json

All GET requests return the following parameters: - guid: Unique identifier of the dashboard, - title: Resource title - description: Resource description - category_name: Name of the category - endpoint: URL pointing to where the resource is located. - tags : Array of tags separated by comma. Optional. - user : Username of resource publisher. - parameters : Null by default. Used to mantain regular structure with other data resources. - created_at : Timestamp of resource revision creation. - link : Link to the resource in the Open Data portal

For example

returns

{
  "result": null,
  "endpoint": "file://73121/71821/147721638324488614901557308458266348545",
  "description": "Investment data at current prices in local currency (millions), USD (millions) and % of GDP",
  "parameters": null,
  "tags": ["InfraLatam", "Infrastructure"],
  "timestamp": null,
  "created_at": "2017-10-02T17:04:02Z",
  "title": "[InfraLatam] Infrastructure total - sum from all infrastructure sectors",
  "modified_at": "2017-10-02T17:16:17Z",
  "category_id": 83353,
  "sources": [{
    "source__id": 2109,
    "source__name": "InfraLatam",
    "source__url": "http://infralatam.info/"
  }],
  "frequency": "",
  "link": null,
  "user": null,
  "guid": "INFRA-INFRA-TOTAL-SUM-FROM",
  "category_name": "Default Category"
}

Automating Dataset Publishing using the Junar API

If you are the owner of a private auth_key, you can automate the data publishing and/or data update process by using our API. You can create or update datasets from local files (which are pushed to the Junar storage servers) or you can define a new, absolute route to where your files are hosted.

The publishing methods support POST/PUT/PATCH requests and expects the following parameters :

POST  /api/v2/datasets.json
PUT   /api/v2/datasets/:guid.json
PATCH /api/v2/datasets/:guid.json
  • title : Title of the dataset. Max. 100 characters.
  • description : Description of the dataset. Max. 250 characters.
  • category : Name of the category where the dataset will be assigned. Must be equal to a category already existing in the account.
  • notes : Extended field for a more complete description of the dataset. Can contain enriched text in HTML (escaped), and supports up to 10.000 characters. Optional.
  • end_point : Absolute URL pointing to where the source file or webpage is located. Mandatory if file hasn’t been added.
  • file : Route to the file to be uploaded onto the Junar storage servers. Mandatory if end_point hasn’t been added.
  • license : Licencing associated to this dataset. Optional
  • spatial : Text field indicating the geographic zone where the dataset applies. Max. 100 characters. Optional.
  • frequency : Dataset resource expected update frequency. Optional.
  • mbox : Email address of the person responsible for data upkeep. Optional.
  • tags : A list of comma separated tags. Optional
  • status: Status of the dataset. Optional.

Possible values for the license field:

- http://creativecommons.org/licenses/by/4.0/
- http://creativecommons.org/licenses/by-sa/4.0/
- http://www.gnu.org/licenses/fdl-1.3.en.html
- http://opendefinition.org/licenses/odc-pddl/
- http://opendatacommons.org/licenses/by/
- http://opendatacommons.org/licenses/odbl/
- http://creativecommons.org/publicdomain/zero/1.0/

If you require a new licence to be added, please contact us at support@junar.com

Possible values for the frequency field:

- yearly
- monthly
- weekly
- daily
- hourly
- ondemand

The possible values for the status field are:

  • published
  • draft
  • pending_review

If the parameter is not sent the default value is pending_review.

If successful, a sample return from the API looks like this:

{
"result": null,
"endpoint": "file://1995/46721/71341786542282142096488420671282999110",
"description": "res",
"parameters": null,
"tags": [ "" ],
"created_at": "2016-02-10T17:10:39",
"title": "resto",
"link": null,
"user": "junarcity",
"guid": "RESTO",
"category_name": "Financial"
}

When updating a dataset, you can change everything from the title and description to the filename and even the file format. Just be sure that, if there’s any resource asociated to this dataset, changes on the data structure will affect it’s outcome. While this in general poses no problem in terms of new records/rows, any new column added will not be represented on pre-existing data views as they are linked to the previous data structure and will not reflect new columns. The data view will have to be edited on the workspace to include any new columns added on a dataset update process.

POST method example

The POST method must be used when creating a new dataset.

  curl -X  POST \
http://demo.cloudapi.junar.com/api/v2/datasets.json \
-H 'cache-control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-H 'postman-token: 58b98775-8b83-4f7e-ed39-cd2225a586f5' \
-F auth_key=238415d936545e1994476b900d41db07883eff72 \
-F 'title=Uploaded from API' \
-F description=Uploaded from API using post method \
-F category=GIS \
-F file=@example.csv

In case the dataset has been successfully created, we will receive this response:

{
  "result": null,
  "endpoint": "file://72121/15031/128335633035380119355724536018989239175",
  "description": "postman",
  "parameters": null,
  "tags": [],
  "timestamp": null,
  "created_at": "2017-06-16T15:20:38Z",
  "title": "Uploaded from API",
  "modified_at": "2017-06-16T15:20:38Z",
  "category_id": 83274,
  "link": null,
  "user": null,
  "guid": "UPLOA-FROM-API",
  "category_name": "GIS"
}

PUT method example

The PUT method is used to update a data set already created. This method requires the sending of all values. So, if a value is missing for the required fields it will return error. In case the field is not obligatory it will send it empty.

  curl -X PUT \
http://junarops.cloudapi.junar.com/api/v2/datasets/SUBID-DESDE-API.json \
-H 'cache-control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-H 'postman-token: 5ff0caf2-6639-07dc-9c70-c01ba59bac9a' \
-F auth_key=238415d936545e1994476b900d41db07883eff72 \
-F 'title=New title' \
-F 'description=This dataset had been updated using put method' \
-F category=GIS \
-F file=@example.xlsx

In case the dataset has been successfully update, we will receive this response:

{
  "result": null,
  "endpoint": "file://72121/15031/272512842402740134577364585136929799929",
  "description": "This dataset had been updated using put method",
  "parameters": null,
  "tags": [],
  "timestamp": null,
  "created_at": "2017-06-12T13:15:06Z",
  "title": "New title",
  "modified_at": "2017-06-16T17:31:02Z",
  "category_id": 83274,
  "link": null,
  "user": null,
  "guid": "UPLOA-FROM-API",
  "category_name": "GIS"
  }

PATCH method example

The PATCH method is used to update a data set already created. In this case only the values to be changed must be sent.The rest of the values are maintained.

In this example we will update only the dataset’s decription:

curl -X PATCH \
http://demo.cloudapi.junar.com/api/v2/datasets/SUBID-DESDE-API.json \
-H 'cache-control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-H 'postman-token: 0527ec7f-f4ac-3bd6-a5f7-9e521076347e' \
-F auth_key=238415d936545e1994476b900d447407883eff72 \
-F description=Updating description using patch method \

In case the dataset has been successfully update, we will receive this response:

{
  "result": null,
  "endpoint": "file://72121/15031/272512842402740134577364585136929799929",
  "description": "Updating description using patch method",
  "parameters": null,
  "tags": [],
  "timestamp": null,
  "created_at": "2017-06-12T13:15:06Z",
  "title": "New title",
  "modified_at": "2017-06-16T17:31:02Z",
  "category_id": 83274,
  "link": null,
  "user": null,
  "guid": "UPLOA-FROM-API",
  "category_name": "GIS"
  }