Untyped "dataspects Software Architecture"

From dataspects::Wiki
C0089421724
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.



DataspectsMediaWikiFeeder

DataspectsMediaWikiFeeder sends to mediawiki-datastore/<mediawiki-id>/pages an entityDoc JSON containing:

wikiPage.string fields
a wikiPage.content hash
  wikiPage.content.string fields
a wikiPage.categories.array of strings
a wikiPage.annotations.hash
  annotation.string fields
  annotation.object.hash
    annotation.object.string fields

mediawiki-datastore

mediawiki-datastore/<mediawiki-id>/pages receives an entityDoc JSON and stores to MongoDB:WikiPage:

wikiPage.string fields
wikiPage.categories array

MongoDB

and stores to MongoDB:WikiPageAnnotation the wikiPage.annotations.hash:

annotation.string fields
annotation.object hash
  object.string fields

and stores to MongoDB:WikiPageAnnotation the wikiPage.categories.array: This could be refactored back into the feeder! See here

annotation.string fields
annotation.object hash
  object.string fields

and synthesizes and stores to MongoDB:WikiPageAnnotation standardTemplateAnnotations and resultFormatTemplateAnnotations annotations.

nats.publish('mediawiki.indexer.indexPage')

{
  wikiPage: wikiPage,
  annotations: annotations
}

mediawiki-workbench

mediawiki-workbench sends to mediawiki-datastore/<mediawiki-id>/pages an entityDoc JSON containing:

type MWResponse struct {
  Source string `json:"source"`
  User string `json:"user"`
  Query string `json:"query"`
  Vol int `json:"vol"`
  Results []Result `json:"results"`
}
type Result struct {
  Meta string `json:"meta"`
  ResourceSiloType string `json:"resourceSiloType"`
  ResourceSiloLabel string `json:"resourceSiloLabel"`
  ResourceSiloID string `json:"resourceSiloID"`
  ResourceType string `json:"resourceType"`
  Pagename string `json:"pagename"`
  RawURL string `json:"rawUrl"`
  ShortURL string `json:"shortUrl"`
  Namespace int64 `json:"namespace"`
  Full Content `json:"full"`
  Annotations []Annotation `json:"annotations"`
}
type Content struct {
  Wikitext string `json:"wikitext"`
  Text string `json:"text"`
  HTML string `json:"html"`
}
type Annotation struct {
  Subject string `json:"subject"`
  Predicate string `json:"predicate"`
  Object Object `json:"object"`
  SMWPropertyType int64 `json:"smwpropertytype"`
}
type Object struct {
  Source string `json:"source"`
  HTML string `json:"html"`
  Text string `json:"text"`
}