Lumada Data Catalog REST API Reference: Virtual Folders

Endpoints under /virtualfolder provide access to VirtualFolder objects. Virtual folders allow users to create groups of resources belonging to a data source into smaller units for easier management. In addition, Virtual folders allow data resources to be part of multiple folders thus letting customers create folders with overlapping sets of data resources.

GET /v2/virtualfolder

A GET request to the /virtualfolder endpoint retrieves a list of all virtual folders configured in the catalog, by default virtual folders will be sorted based on name in ascending order. A VirtualFolder object includes:

The virtual folder "name", a string set by the creator to identify the data source.
A description of the data source visible to users in the Waterline Data UI.
A Parent VirtualFolder whose child this virtual folder will be.
A PathSpecification which defines the scope of VirtualFolder in terms of path, includeExpression and excludeExpression.

Request Parameters
name	type	description	default
browse	query		false
size	query		25
sortColumn	query	supported columns name, time_of_creation, time_of_last_change. Default column "name"	name
sortDirection	query	supported sort direction values asc/ASC & desc/DESC, default asc	asc
start	query		0

Request Body
media type	data type	description
application/json	object (JSON)	A GET request to the `/virtualfolder` endpoint retrieves a list of all virtual folders configured in the catalog, by default virtual folders will be sorted based on name in ascending order. A VirtualFolder object includes: The virtual folder "name", a string set by the creator to identify the data source. A description of the data source visible to users in the Waterline Data UI. A Parent VirtualFolder whose child this virtual folder will be. A PathSpecification which defines the scope of VirtualFolder in terms of path, includeExpression and excludeExpression.

Response Body
media type	data type	description
application/json	PaginatedResponseOfVirtualFolder (JSON)	A list of VirtualFolder objects in JSON format.

Example

Request

GET /v2/virtualfolder
Content-Type: application/json
Accept: application/json

                
{
  "name" : "MyFolder",
  "description" : "all spreadsheets",
  "parentKey" : "{parentVFKey}",
  "pathSpecifications" : [ {
    "sourcePath" : "/user/cloudera/demo-data-19/pub/insurance/countries/us",
    "excludePattern" : "",
    "includePattern" : ".*csv"
  } ]
}

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
{
  "list" : [ {
    "rootFolder" : true,
    "logicalFolderContext" : "...",
    "canRunJob" : true,
    "dataSourceRef" : "...",
    "parentKey" : "...",
    "name" : "...",
    "experts" : [ "...", "..." ],
    "pathSpecifications" : [ {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    }, {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    } ],
    "dataSourceName" : "...",
    "skipValidateVF" : true,
    "resourcePath" : "...",
    "systemName" : "...",
    "timeOfLastChange" : 12345,
    "timeOfCreation" : 12345,
    "key" : "...",
    "description" : "...",
    "type" : "...",
    "restricted" : [ "...", "..." ],
    "viewAble" : [ "...", "..." ],
    "updateable" : [ "...", "..." ],
    "richDescription" : "...",
    "score" : 12345.0,
    "customProperties" : [ {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    }, {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    } ]
  }, {
    "rootFolder" : true,
    "logicalFolderContext" : "...",
    "canRunJob" : true,
    "dataSourceRef" : "...",
    "parentKey" : "...",
    "name" : "...",
    "experts" : [ "...", "..." ],
    "pathSpecifications" : [ {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    }, {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    } ],
    "dataSourceName" : "...",
    "skipValidateVF" : true,
    "resourcePath" : "...",
    "systemName" : "...",
    "timeOfLastChange" : 12345,
    "timeOfCreation" : 12345,
    "key" : "...",
    "description" : "...",
    "type" : "...",
    "restricted" : [ "...", "..." ],
    "viewAble" : [ "...", "..." ],
    "updateable" : [ "...", "..." ],
    "richDescription" : "...",
    "score" : 12345.0,
    "customProperties" : [ {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    }, {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    } ]
  } ],
  "totalCount" : 12345
}

POST /v2/virtualfolder

A POST request to the /virtualfolder endpoint updates a VirtualFolder object.

Request Body
media type	data type
application/json	VirtualFolder (JSON)

Response Body
media type	data type	description
application/json	VirtualFolder (JSON)

Example

Request

POST /v2/virtualfolder
Content-Type: application/json
Accept: application/json

                
{
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
}

Response

HTTP/1.1 201 Created
Content-Type: application/json

                
{
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
}

GET /v2/virtualfolder/databases

A GET request to fetch list of hive databases from the specified Virtual Folder.

Request Parameters
name	type	description
virtualFolderName	query	A (hive) datasource name

Response Body
media type	data type	description
application/json	array of string (JSON)	List of databases or null if error is encountered

Example

Request

GET /v2/virtualfolder/databases
Content-Type: */*
Accept: application/json

                
...

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
...

DELETE /v2/virtualfolder/{key}

A DELETE request to the /virtualFolder/{key} endpoint deletes VirtualFolder and all dependent entities of that virtual folder.

Request Parameters
name	type	description
key	path	key of hte virtual-folder

Example

Request

DELETE /v2/virtualfolder/{key}
Content-Type: */*

                
...

Response

HTTP/1.1 204 No Content

GET /v2/virtualfolder/{key}

A GET request to the /virtualfolder/{key} endpoint retrieves details of a specific virtual folder.

Request Parameters
name	type	description
key	path	The identifying key of a virtual folder, as returned in the list from a call to listVirtualFolders().

Response Body
media type	data type	description
application/json	VirtualFolder (JSON)	The VirtualFolder object in JSON format.

Example

Request

GET /v2/virtualfolder/{key}
Content-Type: */*
Accept: application/json

                
...

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
{
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
}

PUT /v2/virtualfolder/{key}

A PUT request to the /virtualFolder/{key} endpoint updates properties of a VirtualFolder object.

Request Parameters
name	type	description
key	path	The unique identifier of the VirtualFolder object.

Request Body
media type	data type	description
application/json	VirtualFolder (JSON)	The modified VirtualFolder in JSON format.

Response Body
media type	data type	description
application/json	VirtualFolder (JSON)	The updated VirtualFolder object in JSON format.

Example

Request

PUT /v2/virtualfolder/{key}
Content-Type: application/json
Accept: application/json

                
{
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
}

Response

HTTP/1.1 204 No Content
Content-Type: application/json

                
{
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
}

GET /v2/virtualfolder/datasource/{datasourceKey}

A GET request to the /virtualfolder/datasource/{datasourceKey} endpoint retrieves a list of all virtual folders under the given data source, by default virtual folders will be sorted based on name in ascending order.

Request Parameters
name	type	description	default
datasourceKey	path	datasource
size	query	page size	25
sortColumn	query	supported columns name, time_of_creation, time_of_last_change. Default column "name"	name
sortDirection	query	supported sort direction values asc/ASC & desc/DESC, default asc	asc
start	query	index of paginated items	0

Response Body
media type	data type	description
application/json	PaginatedResponseOfVirtualFolder (JSON)	A list of VirtualFolder objects in JSON format.

Example

Request

GET /v2/virtualfolder/datasource/{datasourceKey}
Content-Type: */*
Accept: application/json

                
...

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
{
  "list" : [ {
    "rootFolder" : true,
    "logicalFolderContext" : "...",
    "canRunJob" : true,
    "dataSourceRef" : "...",
    "parentKey" : "...",
    "name" : "...",
    "experts" : [ "...", "..." ],
    "pathSpecifications" : [ {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    }, {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    } ],
    "dataSourceName" : "...",
    "skipValidateVF" : true,
    "resourcePath" : "...",
    "systemName" : "...",
    "timeOfLastChange" : 12345,
    "timeOfCreation" : 12345,
    "key" : "...",
    "description" : "...",
    "type" : "...",
    "restricted" : [ "...", "..." ],
    "viewAble" : [ "...", "..." ],
    "updateable" : [ "...", "..." ],
    "richDescription" : "...",
    "score" : 12345.0,
    "customProperties" : [ {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    }, {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    } ]
  }, {
    "rootFolder" : true,
    "logicalFolderContext" : "...",
    "canRunJob" : true,
    "dataSourceRef" : "...",
    "parentKey" : "...",
    "name" : "...",
    "experts" : [ "...", "..." ],
    "pathSpecifications" : [ {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    }, {
      "sourcePath" : "...",
      "excludePattern" : "...",
      "includePattern" : "..."
    } ],
    "dataSourceName" : "...",
    "skipValidateVF" : true,
    "resourcePath" : "...",
    "systemName" : "...",
    "timeOfLastChange" : 12345,
    "timeOfCreation" : 12345,
    "key" : "...",
    "description" : "...",
    "type" : "...",
    "restricted" : [ "...", "..." ],
    "viewAble" : [ "...", "..." ],
    "updateable" : [ "...", "..." ],
    "richDescription" : "...",
    "score" : 12345.0,
    "customProperties" : [ {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    }, {
      "description" : "...",
      "value" : { },
      "facetAble" : true,
      "name" : "...",
      "type" : "...",
      "displayName" : "..."
    } ]
  } ],
  "totalCount" : 12345
}

GET /v2/virtualfolder/dependencies/{key}

A GET request to the /virtualFolder/dependencies/{key} endpoint returns count of various dependent entities of that virtual folder.

Request Parameters
name	type	description
key	path

Response Body
media type	data type	description
application/json	VirtualFolderDependencies (JSON)

Example

Request

GET /v2/virtualfolder/dependencies/{key}
Content-Type: */*
Accept: application/json

                
...

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
{
  "dataSetCount" : 12345,
  "childVirtualFolderCount" : 12345
}

GET /v2/virtualfolder/list/hive

List all available virtual folder with permissions and source type hive

Response Body
media type	data type	description
application/json	array of VirtualFolder (JSON)

Example

Request

GET /v2/virtualfolder/list/hive
Content-Type: */*
Accept: application/json

                
...

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
[ {
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
} ]

GET /v2/virtualfolder/name/{name}

A GET request to the /virtualfolder/name endpoint retrieves details of a specific virtual folder.

Request Parameters
name	type	description
name	path	A virtual folder name string.

Response Body
media type	data type	description
application/json	VirtualFolder (JSON)	The VirtualFolder object in JSON format.

Example

Request

GET /v2/virtualfolder/name/{name}
Content-Type: */*
Accept: application/json

                
...

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
{
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
}

GET /v2/virtualfolder/list/hive/agent

List all available hive source type virtual folders with permissions from the agent, for which the resource passed belongs to.

Request Parameters
name	type	description
resourceKey	query

Response Body
media type	data type	description
application/json	array of VirtualFolder (JSON)

Example

Request

GET /v2/virtualfolder/list/hive/agent
Content-Type: */*
Accept: application/json

                
...

Response

HTTP/1.1 200 OK
Content-Type: application/json

                
[ {
  "rootFolder" : true,
  "logicalFolderContext" : "...",
  "canRunJob" : true,
  "dataSourceRef" : "...",
  "parentKey" : "...",
  "name" : "...",
  "experts" : [ "...", "..." ],
  "pathSpecifications" : [ {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  }, {
    "sourcePath" : "...",
    "excludePattern" : "...",
    "includePattern" : "..."
  } ],
  "dataSourceName" : "...",
  "skipValidateVF" : true,
  "resourcePath" : "...",
  "systemName" : "...",
  "timeOfLastChange" : 12345,
  "timeOfCreation" : 12345,
  "key" : "...",
  "description" : "...",
  "type" : "...",
  "restricted" : [ "...", "..." ],
  "viewAble" : [ "...", "..." ],
  "updateable" : [ "...", "..." ],
  "richDescription" : "...",
  "score" : 12345.0,
  "customProperties" : [ {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  }, {
    "description" : "...",
    "value" : { },
    "facetAble" : true,
    "name" : "...",
    "type" : "...",
    "displayName" : "..."
  } ]
} ]

Virtual Folders Resource

GET /v2/virtualfolder

Example

Request

Response

POST /v2/virtualfolder

Example

Request

Response

GET /v2/virtualfolder/databases

Example

Request

Response

DELETE /v2/virtualfolder/{key}

Example

Request

Response

GET /v2/virtualfolder/{key}

Example

Request

Response

PUT /v2/virtualfolder/{key}

Example

Request

Response

GET /v2/virtualfolder/datasource/{datasourceKey}

Example

Request

Response

GET /v2/virtualfolder/dependencies/{key}

Example

Request

Response

GET /v2/virtualfolder/list/hive

Example

Request

Response

GET /v2/virtualfolder/name/{name}

Example

Request

Response

GET /v2/virtualfolder/list/hive/agent

Example

Request

Response