View source for C190110121700

{{Setup
|HasEntityTitle=dataspectsSystem on DigitalOcean
|HasEntityBlurb=This setup explains how dataspectsSystem is set up on dataspects' own DigitalOcean infrastructure.
}}
== docker-compose.yml ==

{{code|/usr/local/etc/dataspects/docker-compose.yml}} '''specifies''' all '''Docker containers and volumes'''.

== Docker volumes ==

The Docker volume {{code|dataspects_webservice_customer_data}} is '''volumed into''' the {{code|webservice}} and {{code|sidekiq}} services at {{code|/usr/dataspectsSoftware/}} inside the containers.

== Images and containers ==

Both '''containers''' for services {{code|webservice}} and {{code|sidekiq}}:

# are '''run from the [[C190110142823]]'s Docker image''' hosted at {{code|550143289598.dkr.ecr.eu-central-1.amazonaws.com/dataspects-webservice}} which '''contains the Ruby gem''' https://github.com/dataspects/dataspects
# have these '''environment variables''' set:
#* {{code|DATASPECTS_SEARCH_CONFIG_FILE: /usr/dataspectsSoftware/dataspectsSearch_config.yml}}
#* {{code|DATASPECTS_PLUGINS_FOLDER: /usr/dataspectsSoftware/PLUGINS}}

== Indexing ResourceSilos ==

{{code|DATASPECTS_PLUGINS_FOLDER: /usr/dataspectsSoftware/PLUGINS}} is the location where '''plugins''' are placed. Code in such plugins '''add and/or overwrite methods''' in https://github.com/dataspects/dataspects.

'''Plugins''' can be:
* code that '''specifies query/result functionality/design''', e.g. https://github.com/dataspects/webservice_search_plugin
* code that '''indexes ResourceSilos''', e.g. https://github.com/dataspects/ProvisionDSAsDSCookbook ([[C180913155810]])

=== Indexing file system directories ===

See [[C0023741356]].

=== Indexing ResourceSilos providing an API ===

ResourceSilos that provide an API are '''indexed by running a {{DomainTerm|dataspects workload script|dataspects}}''' by [[C190909102000]].

Here's an example of a workload script:

<syntaxhighlight lang="Bash">
user@workstation:~/myIndexingWorkspace$ tree
.
├── profiles.yml
└── myIndexer.rb
</syntaxhighlight>

<syntaxhighlight lang="Ruby">
# myIndexer.rb
module Dataspects
  class MyIndexer < Indexer

    # STEP 1: Specify basic indexing components
    def initialize
      @sProfilesURL = 'profiles.yml'
      # The following labels refer items in profiles.yml
        @sTIKAServerLabel = 'dataspectsSystemTIKAServer'
        @sElasticsearchClusterName = "dataspectsSystemESCluster"
        @sResourceSiloName = "localmediawiki"
      @sIndexName = "smwckindex-0"
      super
      # Specify a ResourceSilo, e.g. a MediaWiki
      @oSMW = SemanticMediaWiki.new(@oProfiles, @sResourceSiloName, @hOptions)
    end

    # STEP 2: Specify which Resources to index from the
    #         ResourceSilo specified to index
    def ajsonObjectURIs
    end

    # STEP 3: Iterate through all Resources selected for indexing
    #         and apply entitizers (splitting a Resource into a single
    #         or multiple entities) and documentors (creating an
    #         ElasticsearchJSONDocument from an Entity)
    def store_RESOURCENAME(jsonResourceURI)
    end

  end
end
</syntaxhighlight>

== Federated search engine ==

{{code|webservice}} '''serves the endpoints''' specified in https://github.com/dataspects/dataspects-webservice/blob/master/config/routes.rb at https://webservice.dataspects.com.

{{Annotation
|AnnotationPredicate=IsAnInstanceOf
|AnnotationObject=C0169948125
}}