SIGINT DSL

From Dataspects
Jump to: navigation, search
https://wiki.dataspects.com/wiki/C1361832955
dataspectsavatar.png

There is index-time and search-time SIGINT.

Index-time resource-silo-specific SIGINT (e.g. https://search.dataspects.com/app/mediawikis) means: 1) converting resources from resource silos to entities (the things that show up as individual search results), 2) extending the entities' native annotations with synthetic annotations (for EPPO LEX2004241024 and HYDRATION LEX200122142917), 3) compiling a normalized Elasticsearch document sent to a specific index.

Implementations of SIGINT tasks and actions are provided either

  • by a dataspectsd Go module/package or
  • by a Benthos workflow wrapped in a simple dataspects SIGINT DSL statement

Example:

eppo:
    mappings:
        HasEntityType:
            fromAnnotation:
                - havingPredicate: HasEntityType
                - havingPredicate: IsInNamespace
                  ifObjectValueIsOneOf: [ Property, Template ]
                - boolAND:
                      ifAnnotationPredicateValueOneOf: [ HasCategory ]
                      ifAnnotationObjectValueOneOf: [ EntityType ]
                  setTo:
                      string: EntityType
            defaultString: MediaWikiPage
        HasEntityTitle:
            fromAnnotation:
                - havingPredicate: HasEntityTitle
            defaultString: 
        HasEntityBlurb:
            fromAnnotation:
                - havingPredicate: HasEntityBlurb
            defaultString: 
tasks:
    - name: Check pages for syntaxhighlighting
      actions:
            wikitextContainingString:
                - string: syntaxhighlight
                  facetTag: syntaxhighlighting
                - string: docker
                  facetTag: docker
    - name: Check for floating properties
      actions:
            wikitextHavingOneOfTheseTemplates:
                - templates: [ Property ]
                  facetTag: FloatingProperty

dataspectsPresentation-Architecture.png