Lumada Data Catalog REST API

Lumada Data Catalog provides a REST API to access metadata held in the catalog. The same API allows applications to insert metadata such as property values, tags, tag associations, and lineage relationships. The API provides access to the same operations available from the Data Catalog browser application.

The API uses JSON objects as request and response payloads. The HTTP call returns a general pass/fail status message; calls that fail at the Data Catalog server return an failure response message with an error code and more detailed message.

The API accepts the same user credentials as the Data Catalog browser application. Before sending API calls, an application sends an authentication request. The server responds to a successful request with a session cookie, which the application then uses in the header of API calls. The token is valid for the length of the session.

Resources

name path methods description
Agent
  • /v2/agent
  • /v2/agent/agent-config-params
  • /v2/agent/register
  • /v2/agent/{key}
  • GET POST PUT
  • GET
  • POST
  • DELETE
Endpoints under /agent provide access to Agent objects. Agents are the processing engine components that can run on multiple remote clusters.
Audit Events
  • /v2/auditevent/list
  • /v2/auditevent/search
  • /v2/auditevent/{key}
  • GET
  • POST
  • GET
Endpoints under /auditevent provide access to audit events—Waterline Data system and user activities. You can use audit event characteristics such as user, date, or type of activity to filter the events returned.
Authentication
  • /v2/login
  • /v2/logout
  • POST
  • GET
The Authentication API accepts the same user credentials as the Waterline Data browser. These API calls control the creation and destruction of a session cookie to validate other API calls.
Browse: Field-level
  • /v2/browse/field/facet
  • POST
Endpoints under /browse/field manage the facets available when users view a list of fields for a given resource.
Browse: Resource-level
  • /v2/browse/allvalues
  • /v2/browse/facet
  • /v2/browse/new
  • /v2/browse/page
  • /v2/browse/pattern
  • /v2/browse/patterntopbottom
  • POST
  • POST
  • POST
  • POST
  • POST
  • POST
Endpoints under /browse allow you to list and manage metadata for data resources based commonalities of location. Compare with the /search functionality, that lists data resources which share some metadata characteristics. Browse functionality depends on how data is stored and its location on a file system or database.
ComponentNetworkStatus
  • /v2/network
  • GET
Configuration
  • /v2/config
  • /v2/config/{name}
  • GET
  • POST
Endpoints under /config provide access to the configuration options that control job and web server behavior.
Custom Header
  • /v2/customheader
  • GET
The Custom Header API returns the customised header and footer details.
Data Objects
  • /v2/dataobject
  • /v2/dataobject/dataObjectExpert
  • /v2/dataobject/dataObjectOrigin
  • /v2/dataobject/dataObjectSensitivity
  • /v2/dataobject/name
  • /v2/dataobject/recommendations
  • /v2/dataobject/searchresource
  • /v2/dataobject/{key}
  • /v2/dataobject/{key}/profile
  • /v2/dataobject/{key}/tag
  • /v2/dataobject/{key}/join/add
  • /v2/dataobject/{key}/join/refresh
  • /v2/dataobject/{key}/join/remove
  • /v2/dataobject/{key}/resource/add
  • /v2/dataobject/{key}/resource/remove
  • GET POST
  • GET
  • GET
  • GET
  • GET
  • GET
  • GET
  • DELETE GET PUT
  • POST
  • POST
  • POST
  • POST
  • POST
  • POST
  • POST
Data Preview
  • /v2/content/{resourceKey}
  • GET
Endpoints under /datapreview provide access to previews of data resource content based on the authorization of the current user.
Data Resources
  • /v2/dataresource
  • /v2/dataresource/fields
  • /v2/dataresource/hivecolumns
  • /v2/dataresource/metadata
  • /v2/dataresource/origin
  • /v2/dataresource/popularity
  • /v2/dataresource/savepatterns
  • /v2/dataresource/{key}
  • /v2/dataresource/browse/{resourceKey}
  • /v2/dataresource/schema/{resourceKey}
  • /v2/dataresource/source/{sourceName}
  • /v2/dataresource/table/{resourceKey}
  • /v2/dataresource/updateschema/{virtualFolderName}
  • /v2/dataresource/view/{resourceKey}
  • /v2/dataresource/virtualfolder/{virtualFolderName}
  • /v2/dataresource/{resourceKey}/partitions
  • /v2/dataresource/{resourceKey}/relatedresources
  • PUT
  • GET POST
  • GET
  • GET
  • GET
  • GET
  • POST
  • GET PUT
  • GET
  • PUT
  • GET
  • POST
  • PUT
  • POST
  • GET
  • GET
  • GET
Endpoints under /dataresource provide access to DataResource objects in the Waterline Data repository. Waterline Data defines a data resource as an HDFS file or folder, a set of files that are organized in folders (known as a Waterline Data collection), as a Hive database or table, or as a relational database or table. DataResource objects are uniquely identified by a "key" value, assigned on creation.
Data Sets
  • /v2/dataset
  • /v2/dataset/name
  • /v2/dataset/{key}
  • /v2/dataset/members/add
  • /v2/dataset/members/remove
  • /v2/dataset/schema/{key}
  • /v2/dataset/{key}/virtualfolder
  • /v2/dataset/schema/version/max
  • GET POST
  • GET
  • DELETE GET PUT
  • POST
  • POST
  • PUT
  • GET
  • GET
Endpoints under /dataset provide access to Virtual Dataset objects. A Waterline Virtual Data Set allow users to create groups of resources having the same schema and spanning different folders in your data lake, into a single virtual unit for easier management. Virtual Data Sets are considered as user defined virtual collections that have the matching schema but may have different path specifications/hierarchy with respect to a data source.
Data Sources
  • /v2/datasource
  • /v2/datasource/list-configs
  • /v2/datasource/validate
  • /v2/datasource/{key}
  • /v2/datasource/dependencies/{key}
  • /v2/datasource/gethint/{datasourcetype}
  • /v2/datasource/name/{name}
  • /v2/datasource/sync/{key}
  • /v2/datasource/credentials/users/{user}
  • /v2/datasource/{key}/credentials/users/{user}
  • GET POST
  • GET
  • POST
  • DELETE GET PUT
  • GET
  • GET
  • GET
  • DELETE
  • GET
  • DELETE POST PUT
Endpoints under /datasource provide access to DataSource objects. A Waterline Data data source is the location of an HDFS directory, a Hive instance or specific database, a cloud storage location, or a relational database instance or specific database. A data source hosts data resources, in the form of folders, files, databases, and tables.
DataOps
  • /v2/dataops/catalogmetrics
  • /v2/dataops/communitymetrics
  • /v2/dataops/keymetrics
  • /v2/dataops/metadataobjectmetrics
  • /v2/dataops/usagemetrics
  • GET
  • GET
  • GET
  • GET
  • GET
Endpoints under /dataops provides following details Number of top level resources, including collections and data sets, Total number of fields in the data lake, All tag associations, at all levels. All significant objects hosted by WDC Count up all items curated or authored by the community All users that we have seen on WDC Total number of searches performed on WDC including catalogmetrics,metadataobjectmetrics,usagemetrics and communitymetrics
Discussions
  • /v2/discussions/comment
  • /v2/discussions/getuser
  • /v2/discussions/topic
  • /v2/discussions/aggregate/{dataresourceKey}
  • /v2/discussions/comment/{commentKey}
  • /v2/discussions/topic/{topicKey}
  • /v2/discussions/comments/parentDiscussionItemKey/{parentDiscussionItemKey}
  • /v2/discussions/comments/resourceKey/{resourceKey}
  • /v2/discussions/resource/{resourceKey}/discussions
  • /v2/discussions/resource/{resourceKey}/topic/allTopics
  • POST
  • GET
  • POST
  • GET
  • DELETE GET PUT
  • DELETE GET PUT
  • GET
  • GET
  • GET
  • GET
Entity Specifications
  • /v2/entityspec/all
  • /v2/entityspec/{name}
  • /v2/entityspec/{name}/propertyspec
  • GET
  • GET
  • POST PUT
An entity specification defines properties for Waterline Data objects such as tags, users, roles, data resources, and so on. Properties of entities are represented by PropertySpec objects. An EntitySpec object associates a set of properties with a specific entity type. Endpoints under /entityspec allow you to create, query, update, and delete properties associated with entities, including setting the access level for properties.
External Sources
  • /v2/externalsource
  • /v2/externalsource/{key}
  • /v2/externalsource/name/{name}
  • GET POST
  • DELETE GET
  • DELETE GET
External Sources are sources of metadata to be imported into the Waterline Data catalog. Endpoints under /externalsource provide a way to identify and connect to these outside applications. For example, the extensions created to import metadata from applications such as Apache Atlas or Cloudera Navigator use external sources to persist connection information for the application instance.
Favorites
  • /v2/favorites
  • /v2/favorites/{entityKey}
  • /v2/favorites/{user}
  • /v2/favorites/{entityKey}/{userName}
  • GET
  • DELETE POST
  • GET
  • POST
Endpoints under /favorites provide a way to manage user bookmarks on tags and resources.
Homepage
  • /v2/homepage/contributions
  • /v2/homepage/my-info
  • /v2/homepage/participation-level
  • /v2/homepage/resources-workedon
  • /v2/homepage/list/dataobjects
  • /v2/homepage/list/hivetables
  • GET
  • GET
  • GET
  • GET
  • GET
  • GET
JobManagement
  • /v2/job/execution
  • /v2/job/execution/terminate/{jobInstanceId}
  • /v2/job/execution/{key}/paginated-details
  • /v2/job/execution/{key}/paginated-resources
  • /v2/job/execution/{key}/resources
  • /v2/job/execution/{key}/virtual-folders
  • POST
  • PUT
  • GET
  • GET
  • GET
  • GET
JobManagement
  • /v2/job/instances
  • /v2/job/instances/currentuser
  • /v2/job/instances/list
  • /v2/job/instances/{key}
  • /v2/job/instances/{key}/executions
  • GET
  • GET
  • DELETE
  • GET
  • GET
JobManagement
  • /v2/job/sequences
  • /v2/job/sequences/{key}
  • GET
  • GET
JobManagement
  • /v2/job/templates
  • /v2/job/templates/bulkdelete
  • /v2/job/templates/{key}
  • /v2/job/templates/{key}/instances
  • GET POST
  • POST
  • DELETE GET PUT
  • GET
Lineage
  • /v2/lineage/addsource
  • /v2/lineage/searchresource
  • /v2/lineage/{key}
  • /v2/lineage/{operationExecutionKey}
  • /v2/lineage/addparent/{key}
  • /v2/lineage/field-mapping-info/{operationExecutionKey}
  • /v2/lineage/hop-count/{key}
  • /v2/lineage/multihop/{key}
  • /v2/lineage/tagAssociations/{key}
  • /v2/lineage/tagDomains/{key}
  • /v2/lineage/edge/{key1}/{key2}/{edgeType}
  • POST
  • GET
  • GET
  • DELETE
  • POST
  • GET
  • GET
  • GET
  • GET
  • GET
  • DELETE
Waterline Data defines a LineageRelations object that records and presents the lineage relationships between data resources. Endpoints under /lineage provide access to these objects.
Metadata Rest Server Config
  • /v2/metadata-server/sync-config
  • POST
Endpoints under /metadata-server provide access to synchronise the metadata rest server configurations to the app-server.
Metadata Server
  • /v2/metadata-server
  • GET
Endpoints under /metadata-server provide access to Metadata Rest Server details objects. Metadata Rest Server provides the endpoints for agents to perform CRUD operations on the repository.
Notification
  • /v2/notification
  • /v2/notification/update
  • /v2/notification/updateAll
  • /v2/notification/{key}
  • GET POST
  • PUT
  • PUT
  • DELETE
Overlap tables
  • /v2/overlap/table
  • /v2/overlap/table/csv
  • GET POST
  • POST
Endpoints under /overlap/table allow you to pass tables information and receive a preliminary overlap report computed based on the table schema
QueryController
  • /v2/query
  • GET
Reviews
  • /v2/reviews
  • /v2/reviews/{key}
  • /v2/reviews/aggregate/{datasourceKey}
  • /v2/reviews/entity/{entityKey}
  • /v2/reviews/review/{reviewKey}
  • /v2/reviews/user/{user}
  • /v2/reviews/users/{user}
  • GET POST PUT
  • DELETE
  • GET
  • GET
  • GET
  • GET
  • POST
Roles
  • /v2/role
  • /v2/role/byname
  • /v2/role/{key}
  • GET POST
  • GET
  • DELETE GET PUT
Waterline Data uses roles to indicate what data resources from the catalog are available to each user. User roles determine what actions each user can perform for metadata management inside Waterline Data. Endpoints under /role allow creation of new roles and modification of existing roles. To associate roles to users, use a PUT request to /user/{key}.
Rules
  • /v2/rule
  • /v2/rule/name
  • /v2/rule/{key}
  • /v2/rule/name/{name}
  • GET POST PUT
  • PUT
  • DELETE GET
  • DELETE GET
Endpoints under /rule allow you to list and manage the rules.
SSLController
  • /v2/ssl/app-server-fingerprint
  • GET
Saved Search
  • /v2/savedsearch
  • /v2/savedsearch/name
  • /v2/savedsearch/{key}
  • GET POST
  • DELETE GET PUT
  • DELETE GET PUT
Endpoints under /savedsearch allow you to list and manage the Saved Searches.
Search
  • /v2/search/facets
  • /v2/search/new
  • /v2/search/page
  • /v2/search/runFacet
  • /v2/search/user
  • GET
  • POST
  • POST
  • POST
  • GET
Endpoints under /search allow you to collect data resources that share some metadata characteristics. Compare the browse functionality, that returns a set of data resources based on the resource's location. Searchable characteristics include facets and keywords.
SessionController
  • /session
  • GET
Tag Associations
  • /v2/tagassociation
  • /v2/tagassociation/byfield
  • /v2/tagassociation/byresource
  • /v2/tagassociation/list
  • /v2/tagassociation/{key}
  • /v2/tagassociation/field-tag-domain-list/{resourceKey}
  • /v2/tagassociation/list/byresource
  • /v2/tagassociation/resource/{key}
  • /v2/tagassociation/tag/{key}
  • /v2/tagassociation/counts/domainlevel/all
  • /v2/tagassociation/tag/container/{key}
  • /v2/tagassociation/counts/taglevel/domain/{key}
  • /v2/tagassociation/counts/taglevel/tag/{key}
  • POST
  • GET
  • GET
  • POST PUT
  • DELETE GET PUT
  • GET
  • POST
  • GET
  • GET
  • GET
  • GET
  • GET
  • GET
Endpoints under /tagassociation provide access to TagAssociation objects in the Waterline Data repository. A tag association object defines the relationship of a tag to a specific data resource (folders, files, collections, and tables) or specific field in a data resource.
Tag Associations Features
  • /v2/tagfeature/tag/{key}
  • /v2/tagfeature/tagassociation/{tagAssociationKey}
  • /v2/tagfeature/tagassociation2/{tagAssociationKey}
  • GET
  • GET
  • GET
Tag Domains
  • /v2/tagdomain
  • /v2/tagdomain/all
  • /v2/tagdomain/alldomains
  • /v2/tagdomain/list
  • /v2/tagdomain/{key}
  • /v2/tagdomain/name/{name}
  • /v2/tagdomain/{domainKey}/alltags
  • POST
  • GET
  • GET
  • POST PUT
  • DELETE GET PUT
  • DELETE GET
  • GET
Waterline Data tags are maintained in a tag glossary that is organized into tag domains. Endpoints under /tagdomain provide access to TagDomain objects.
Tags
  • /v2/tag
  • /v2/tag/byname
  • /v2/tag/hierarchy
  • /v2/tag/list
  • /v2/tag/sensitivity
  • /v2/tag/{key}
  • /v2/tag/children/{key}
  • /v2/tag/regex/validate
  • POST
  • GET
  • POST
  • POST PUT
  • GET
  • DELETE GET PUT
  • GET
  • POST
Endpoints under /tag provide access to Tag objects in the Waterline Data repository. Tags are labels associated with data resources (folders, files, collections, or tables), or with specific fields in data resources.
Tokens
  • /v2/token
  • /v2/token/{clientId}
  • /v2/token/{key}
  • GET POST PUT
  • GET
  • DELETE
Endpoints under /token provide access to Token objects. Tokens are used to authenticate inter component communication for the distributed architecture.
User Profiles
  • /v2/user
  • /v2/user/_me
  • /v2/user/{key}
  • /v2/user/assign/role
  • /v2/user/list/roles
  • /v2/user/name/{username}
  • /v2/user/permit/resource
  • /v2/user/permit/source
  • /v2/user/remove/role
  • GET POST
  • GET PUT
  • DELETE GET PUT
  • POST
  • GET
  • GET
  • GET
  • GET
  • POST
Endpoints under /user allow administrators to manage user profiles, represented by the User object. The caller must have a roles that includes an Administrator access level to create, update, or delete user profiles.
Virtual Folders
  • /v2/virtualfolder
  • /v2/virtualfolder/databases
  • /v2/virtualfolder/{key}
  • /v2/virtualfolder/datasource/{datasourceKey}
  • /v2/virtualfolder/dependencies/{key}
  • /v2/virtualfolder/list/hive
  • /v2/virtualfolder/name/{name}
  • /v2/virtualfolder/list/hive/agent
  • GET POST
  • GET
  • DELETE GET PUT
  • GET
  • GET
  • GET
  • GET
  • GET
Endpoints under /virtualfolder provide access to VirtualFolder objects. Virtual ​folders ​allow ​users ​to ​create ​groups ​of ​resources ​belonging ​to ​a ​data ​source ​into ​smaller ​units ​for easier ​management. ​In ​addition, ​Virtual ​folders ​allow ​data resources ​to ​be ​part ​of ​multiple ​folders ​thus letting ​customers ​create ​folders ​ with ​overlapping ​sets ​of ​data ​resources.

Data Types

JSON

type description
AbstractField
AccessLevel
Agent
AgentConfigurationParams
AllValuesRequest
AllValuesTO
Asset
AssociationCount
AtomicField
AuditEvent
AuditEventListResponse
AuditFilterCriteria
BrowseRequest
BrowseResult
CatalogMetricsTO
CatalogMetricsUnit
Category Enumeration that describes groupings for configuration properties. These categories are for organization and are not functional.
CommunityMetrics
ComponentNetworkStatus
ConfigProperty Type that describes the configuration settings used to control Waterline Data behaviors. Properties include controls such as whether to profile files that failed previously. The information provided for each property includes the property name, the label used in the UI, a description of what the property controls, and attributes for who and how the property can be viewed or changed.
ContributionsCount
CreateAgentResponse
Credential
Credentials Type that describes the username and password for the current user's authenticated session. Credentials details are passed to the system, but cannot be returned.
CurrentUserInformation
CustomHeaderProperties Type that defines the various properties in the custom header properties configuration file
CustomProperty
DataObjectRequest
DataPartition
DataPreview
DataResource
DataResourceFlexTO
DataResourceLite
DataSource
DataSourceDependencies
DataType
DatasetMember
DatasetRequest
DiscussionCommentTO
DiscussionItem
DiscussionItemsMetrics
DiscussionTopic
Edge
Entity
EntitySpecification
EventType
ExecutionMetrics
Experts
ExpertsType
ExtendedMetrics
ExternalSource
FacetResult
FacetSelection
Favorite
FieldNode
IterableOfAgent
IterableOfExternalSource
IterableOfTableComparisionTO
IteratorOfString
JobDetailsTO
JobExecutionResultTO
JobExecutionTO
JobSequenceTO
JoinConditionTO
KeyMetrics
LineageHopCountTO
LineageRelationGraph
LineageRelationPath
LineageRelations
LineageSearchPathTO
LoginResponse Type that provides a response to a login attempt. Successful attempts return a SUCCESS flag and message and echo the username. Failed attempts return a FAILURE flag and a message that describes the reason for failing.
LoginResult Enumeration of the possible outcomes of a login attempt, including "SUCCESS" and "FAILURE".
LogoutResponse
Member
MetadataObjectMetrics
MetadataServer
MetadataServerConnectionParams
Metrics
Node
Node
NodeKeyAndLevelWrapper
NodeType
Notification
NotificationPaginatedResponse
NotificationRequestTO
NotificationType
ObjectField
OpDetails
OperationExecution
OperationNode
OrderbySpec
PaginatedResponse
PaginatedResponse2OfDataResourceFlexTO
PaginatedResponseOfDataResource
PaginatedResponseOfDataResourceLite
PaginatedResponseOfDiscussionCommentTO
PaginatedResponseOfDiscussionTopic
PaginatedResponseOfJobDetailsTO
PaginatedResponseOfPairOfStringAndDataResource
PaginatedResponseOfPairOfStringAndString
PaginatedResponseOfSavedSearch
PaginatedResponseOfVirtualFolder
PagingCriteria
PairOfStringAndDataResource
PairOfStringAndString
Participation
ParticipationLevel
PathElement
PathInfo
PathSpecification
PatternTO
PatternTopBottomTO
PatternValue
PatternValue
PatternValue
PreFilterTO
Principal
Properties
Property
PropertySpecification
PropertyType
QueryRequest
QueryResult
RegexTester
RelatedResourcesMap
ReportedField
ResourceField
ResourceNode
ReviewsAggregate
Role
Rule
RuleAction
RuleScope
SSLDetailsTO
SavedSearch
SearchCriteria
SearchResult
SearchType
Semantic
Semantic
SequenceTO
SessionData
SolrConnectionParams
Source
SparkJob
State
State
StatusDetail
SystemNode
SystemTO
TableComparisionTO
Tag
TagAssociation
TagAssociationContainer
TagAssociationFeature
TagAssociationRequest
TagAssociationView
TagDetail
TagDetails
TagDomain
TagDomainContainer
TagDomainCountByResource
TagDomainCountTO
TagRequest
TagState
Token
UpdatedUserReview
UsageMetrics
User
UserInfo Type that provides information about the current user.
UserReview
UserTO
VirtualFolder
VirtualFolderDependencies
Visibility