Main Page

dataspects is a knowledge management system based on

• a search engine that can index any kind of information source and
• a wiki used as the primary platform to manage knowledge as connected entities organized by the principles of Every Page is Page One
• explained by a distinct story.
• Start "Introduction to FindAndLearn"

How this cookbook is organized

Every piece of knowledge (regarding orientation, understanding, subjects, workflows and procedures) is of a distinct Entity Types ^dataspects. See Dashboard "Enterprise Knowledge Management Aspects (Concerns, Responsibilities, Roles, Use Cases and Tools/Methodologies)".

CanBeDeveloped | MustBeMaintained | CanBeInstalled | CanBeTested

Curation Dataspects Dataspects workload script DataspectsSearch DataspectsSystem DesignChallenge Development Domain Terms Editing Elastic Elasticsearch EntityTypes+Predicates Extensions Faceting Indexing Installation Integration Maintenance Plugins Promotion Query DSL Ruby Support Viewing Wiki

EntityType Entities AAAA-Ingestion Add new AAAA-Ingestion "From E-Mail account to Datastore by" HasIngestionSource::E-Mail account HasIngestionTarget::Datastore HasIngestionType::Feeder AAAA-Ingestion "From End user to File by Editor UI" HasIngestionFeeder::Editor UI HasIngestionSource::End user HasIngestionTarget::File HasIngestionType::UI AAAA-Ingestion "From End user to MediaWiki by E-Mail" HasIngestionFeeder::E-Mail HasIngestionSource::End user HasIngestionTarget::MediaWiki HasIngestionType::UI AAAA-Ingestion "From End user to MediaWiki by MediaWiki Web UI" HasIngestionFeeder::MediaWiki Web UI HasIngestionSource::End user HasIngestionTarget::MediaWiki HasIngestionType::UI AAAA-Ingestion "From File system to Datastore by https://github.com/dataspects/dsdocument-cli" HasIngestionFeeder::https://github.com/dataspects/dsdocument-cli HasIngestionSource::File system HasIngestionTarget::Datastore HasIngestionType::Feeder AAAA-Ingestion "From MediaWiki to Datastore by https://github.com/dataspects/DataspectsMediaWikiFeeder" HasIngestionFeeder::https://github.com/dataspects/DataspectsMediaWikiFeeder HasIngestionSource::MediaWiki HasIngestionTarget::Datastore HasIngestionType::Feeder AAAA-Ingestion "From MediaWiki to Datastore by https://github.com/dataspects/mediawiki-workbench" HasIngestionFeeder::https://github.com/dataspects/mediawiki-workbench HasIngestionSource::MediaWiki HasIngestionTarget::Datastore HasIngestionType::Feeder AAAA-Ingestion "From Repository to dataspectsd by" HasIngestionSource::Repository HasIngestionTarget::dataspectsd HasIngestionType::Feeder Argument Add new Argument "Flexibility of structure" The structure (ontology, "network of relations") is powerful and flexible: While it's a good idea to start with a first structure to import and create initial content, the structure is growing with the wiki. New ideas and necessary changes can be integrated easily and updated with a few clicks. Even more, the growing structure is not just a technical, alimental skeleton, but an integral part of the content in the wiki and an additional source of newly created knowledge. Argument "No vendor lock-in" Argument "Search result views have a unique reference URL" Search results offer a wide range of details (show values, entities, facets) to specify, filter and drilldown results. Each result display/view gets a unique URL in the wiki, so that it can be referenced (i.e. the link forwarded, included in other wiki pages.) Argument "Structured and unstructured content go hand in hand" Content within the structured ontology in the wiki is closely held together by relations (via annotations, properties, categories). This structure is working in the background. The user has additional possibilities to relate content (via links and manually created annotations). Also content that is not prestructured whatsoever can be dropped in the wiki, linked to and integrated into the structure later on. ConcernsAspect::Curation Audience Add new BusinessService Add new BusinessService "Run your organizational Find-And-Learn network!" Knowledge management consists of optimizing the access to and the management of knowledge for the sake of decision and performance support. HasGitHubRepository::https://github.com/dataspects/dataspectsSystem BusinessService "xml-sitemaps.com" Concept Add new Concept "_analyze" HasElasticConceptType::Endpoint RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html Concept "_source" HasESContext::Query HasElasticConceptType::SearchRequestComponent RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering.html Concept "Asymmetric Anaylsis" Analysis applied at index time is different from the analysis applied at query time. Concept "Capture meaning with synonyms" HasElasticConceptType::AnalysisStrategy Concept "CoKe" Efficient retrieval is facilitated by prepending items in structured lists (nesting, bulleting, numbering, letter casing, coloring, linking) using cognitive keywords. This approach is understood as stating knowledge using the programming language CoKe meant to be executed on human brains. Concept "Controlled, faceted, multilateral, asynchronous, bidirectional synchronisation of multiple SMWs (CFMABS)" Concept "Document Text Analysis" Concept "Full text queries > match_phrase (aka proximity)" HasElasticConceptType::QueryType PerformsAnalysisBeforeSearching::True Concept "Inheriting annotations if not customized by Indexer" Understand how to customize ES doc compilation Concept "Integration Testing dataspect Search Gatsby" ds0--HasContext::https://entities.dataspects.com/Testing Concept "IsRootPredicate" dataspects canonical ES index mapping contains domain-agnostic root fields (predicates) and nested annotations that accommodate domain-specific predicates. Concept "Join: has_child and has_parent queries" HasElasticConceptType::QueryType RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/joining-queries.html Concept "Keyword datatype" Index content without analyzing it such as status or zip codes typically used for filtering, sorting and aggregations. Keyword fields are only searchable by their exact value. HasElasticConceptType::Datatype Concept "Knowledge Topics/Subjects/Bites" Concept "Manage Domain Terms" Concept "Modeling specificity" HasElasticConceptType::AnalysisStrategy Concept "Parent-Child Relationship" The join data type is a special field that creates parent/child relation within documents of the same index. HasElasticConceptType::Relationship HasRestriction::The parent document and all of its children must live on the same shard. RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html Concept "Reification" Concept "SIGINT DSL Actions" ds0--HasContext::https://wiki.dataspects.com/wiki/SIGINT_DSL ds0--showAvatar::dataspectsavatar.png Concept "SIGINT" ds0--HasContext::https://entities.dataspects.com/dataspects Concept "Simple Analyzer" * breaks text into terms at non-letter characters all terms lower cased HasElasticConceptType::Analyzer RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simple-analyzer.html Concept "Standard Analyzer" Grammar based tokenization based on the Unicode Text Segmentation algorithm HasElasticConceptType::Analyzer RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-standard-analyzer.html Concept "Text datatype" Index content analyzing it. HasElasticConceptType::Datatype Concept "Unit Testing dataspect Search Gatsby" ds0--HasContext::https://entities.dataspects.com/Testing showAvatar::gatsbyavatar.png Concept "dataspects Knowledge Graph" Concept "full_text/match_phrase_prefix" HasElasticConceptType::QueryType PerformsAnalysisBeforeSearching::True Concept "full_text/query_string" The query_string query parses the input and splits text around operators. Each textual part is analyzed independently of each other. HasElasticConceptType::QueryType PerformsAnalysisBeforeSearching::True Concern Add new Concern "Organize a knowledge management conference" ConcernsAspect::Cooperation Configuration Add new Configuration "Wikitext Search" Wikitext search allows for finding raw wikitext snippets. HasElasticConceptType::FieldsContent-Analysis-Query-Combination Dashboard Add new Dashboard "Enterprise Knowledge Management Aspects (Concerns, Responsibilities, Roles, Use Cases and Tools/Methodologies)" This article explains all aspects (roles, responsibilities, use cases and tools/methodologies) that keep your enterprise knowledge management healthy, effective and efficient. DataspectsSystemModule Add new DataspectsSystemModule "Support" DesignPrinciple Add new Diagram Add new Diagram "dataspectsSearch system backup with duplicity" Explains::Which software packages are necessary HasDrawIoURL::https://drive.google.com/a/dataspects.com/uc?id=0B1LNGPG7-v6OZWlSVl96ZzM3U3M Diagram "dataspectsSearch system module layers diagram" Explains::dataspectsSearch core and custom modules HasDrawIoURL::https://drive.google.com/a/dataspects.com/uc?id=1fsrfNd4W7wAltdDawkvjpY120vQSuWH1 Diagram "dataspectsSystem Dockerized Setup" HasDrawIoURL::https://drive.google.com/a/dataspects.com/uc?id=1VAKkbP3j0m5Q1W1sjBVu3U8evMt26ZkS Feature Add new Feature "Autocomplete did-you-mean completion suggesters" RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html Feature "Automatic ontology dashboard exposure" Feature "Case-insensitive concept disambiguation" Feature "Code Graph" Get a graph from a facet of your code base showing function calls. HasCodeTag::LEX2010051135 Feature "EPPO SIGINT" HasCodeTag::LEX2009300938 ds0:isReleasedDataspectsFeature::true Feature "Element Secure Collaboration and Messaging ESCAM" Feature "EnrichKGByExternalSources" EnrichKGByExternalSourcesTest() Feature "Entitizing" HasCodeTag::LEX2104271021 Feature "Explain a domain (facet)" Explain a (sub)domain as a facet of connected entities. Display the explanation as a graph and a bulleted story. ds0:isReleasedDataspectsFeature::true Feature "Formredlink for AnnotationObject if not exist" ds0:isReleasedDataspectsFeature::true Feature "Go Code Repository SIGINT" HasPendingIssue::VariablesReferToStructMap must also "2) in this function's content whether it assigns <STRING> to a struct." Feature "Index and show Google-Drive-hosted draw.io diagrams annotated by a SMW page of type 'Diagram'" ds0:isReleasedDataspectsFeature::true Feature "Indexing file system directories" Feature "Link to dataspects facets" Use GET-parametrized URLs to link to facets returned as search results, or create a parser function that would look up the facet JSON on SearchFacet pages and then either GET or POST. ds0:isReleasedDataspectsFeature::true Feature "Load annotation object tokens" HasEgrepExplanationCommand::LEX2009071157 Feature "Mail to Knowledge Base" The MailToKnowledgeBase functionality enables sending content to the API via email for automatic semantic integration. Feature "Match natural language queries to predicates" HasEgrepExplanationCommand::LEX2009071151 Feature "NLP" Feature "Nested Facet Maps" Contextualize entities along multiple dimensions simultaneously Feature "SIGINT explanations" Covered by ds100 predicates HasEgrepExplanationCommand::LEX2009091211 Feature "Search and retrieve" HasGitHubRepository::www MentionsDomainTerm::Index Aliases MentionsDomainTerm::Search API MentionsDomainTerm::Search Requests MentionsDomainTerm::Query DSL MentionsDomainTerm::Node MentionsDomainTerm::Filter DSL MentionsDomainTerm::Search Routing MentionsDomainTerm::Search Request Scope Feature "Search engine mode (SEM)-specific search results design, behavior and interaction" Search must optimize itself depending on the search context (represented by a SEM) that the user has in mind. HasCodeTag::LEX2009201433 ds0:isReleasedDataspectsFeature::true Feature "Search result highlighting" RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-highlighting.html ds0:isReleasedDataspectsFeature::true Feature "TextEntitizer" Feature "Token-based Insight" HasCodeTag::LEX2104291408 Feature "searchResult-specific actions" HasEgrepExplanationCommand::./inspection/grep_implementsFeature.sh "\[\[ImplementsFeature::C0332407110" ds0:isReleasedDataspectsFeature::true IndexingSetup Add new IndexingSetup "Indexing a SemanticMediaWiki instance" IndexingSetup "Indexing a file system directory" Dataspects:Issue Add new Onepager Add new Onepager "Arrange your knowledge on the fly" AddressesAudience::User ConcernsAspect::Promotion Onepager "Easy diagram management in the wiki" Diagrams can easily (and visually appealing) be created via draw.io. The diagrams are stored on a server and are seamlessly integrated onto wiki pages and into search results. From search results, the diagrams can be viewed and edited (link to editing page). Like this, the external diagram service is best integrated into the wiki. Onepager "Leash your business knowledge" AddressesAudience::Management ConcernsAspect::Promotion HasGoogleDocFile::https://docs.google.com/document/d/1XaMKEzfdE8tfkV5azDfvUkrKYdZ4vB98r3b_pffm820/edit Onepager "Onepager per role" Onepager per user role in the company and/or ontology – benefits for this user role, responsibilites, working fields in the wiki, chances etc. Ontology Add new Ontology "ConferenceManagementOntology" Ontology "EnterpriseKnowledgeManagementOntology" The pervasion of knowledge in an organization DependsOn::Ontology "dataspectsSystemCoreOntology" Ontology "dataspectsSupportOntology" Manages problem hierarchies and their solutions. HasGitHubRepository::https://github.com/dataspects/dataspectsSupportOntology Ontology "dataspectsSystemCoreOntology" How to store and structure knowledge on a SMW — here's the framework HasGitHubRepository::https://github.com/dataspects/dataspectsSystemCoreOntology HasPresentation::https://www.mediawiki.org/wiki/EMWCon_Spring_2018/Implementing_EPPO_on_SMW Ontology "issueManagementOntology" Manage issues akin to GitHub Presentation Add new Presentation "How to store and structure knowledge on a SMW — here's the framework" AddressesAudience::MWStake ConcernsAspect::Promotion IsCarriedOutByRole::Role "Promoter" Principle Add new Principle "Efficient (Re-)Naming Support (ENS)" Optimal naming of knowledge assets evolves and a system must enable adapting names without breaking things. Principle "Filter when you can, query when you have to." Principle "PredicateClass" Problem Add new Problem "Docker daemon asks for authentication for pulling a public image" HasSolution::Solution "Delete Docker authentication token" Problem "Opcode cache caches signatures" Problem "PHP Composer invalid installer signature" CanBeCausedByProblem::Problem "Opcode cache caches signatures" HasSolution::Solution "Download again and check $HASH" MentionsErrorCode::1234:56 Recipe Add new Recipe "Add draw.io diagrams hosted on Google Drive to a dataspects DSKMF Elastic index" Recipe "Add/edit subject types" ConcernsAspect::Curation HasIdea::Add functionality as special page to https://github.com/dataspects/DSKMFCoreMWExtension IsCarriedOutByRole::Role:PowerUser Recipe "Aliasing" RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-add-alias.html Recipe "Create Kibana visualizations and dashboard" Recipe "Create a saved search in Kibana" Saved searches are a type of saved objects in Kibana. Recipe "Customize a SMW-resource-born entity's sHasResourceURL for indexing" Recipe "Extract/export/inject/import a set of pages from/into a SMW from/into a directory (git repository)" Recipe "Restore a dataspectsSystem from a Duplicity backup set" ConcernsAspect::Installation IsCarriedOutByRole::Role "Administrator" Recipe "Search for property tokens created by mutliple tokenizers" RefersToElasticsearchDocumentation::https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html Recipe "Setup and configure the skin Tweeki" Recipe "Ways to make remote Duplicity backup sets available in a local directory" ConcernsAspect::Installation IsCarriedOutByRole::Role "Administrator" RelevanceEngineeringCase Add new RelevanceEngineeringCase "Make CamelCase HasEntityTitle flexibly findable" ResourceIndexerClass Add new ResourceIndexerClass "WikitextFiles" MustNotUse::ResourceIndexerModule "AnnotationParser" ResourceIndexerModule Add new ResourceIndexerModule "AnnotationParser" ResourceSiloType Add new ResourceSiloType "MediaWiki" ResourceType Add new ResourceType "MediaWikiPage" Responsibility Add new Responsibility "Maintain/develop MW/SMW functionality" Responsibility "Maximize business/use cases for SMW" Responsibility "Maximize data quality" Responsibility "Maximize information coherence" Responsibility "Maximize information source converage" Responsibility "Maximize interface quality" Responsibility "Promote professional KM" Responsibility "Provide KM management infrastructure and tools" Role Add new Role "Administrator" Role "Content Curator" Role "New employee" Role "Ontology Designer" #Facet management Type management Property management Role "Promoter" Role "Reader" Role "Search Engineer" Role "User" #Reader Contributor/Editor Poweruser Gardener RubyGem Add new RubyGem "dataspects" CanBeDeveloped::Yes CanBeInstalled::Yes CanBeTested::Yes Requires::Ruby SearchContext Add new SearchContext "Support" Setup Add new Setup "Dockerized dataspectsSystem" CanBeInstalled::Yes HasQualityAssuredBy::Team "DataspectsLinuxTeam" Setup "dataspectsSystem on DigitalOcean" This setup explains how dataspectsSystem is set up on dataspects' own DigitalOcean infrastructure. IsAnInstanceOf::Setup "Dockerized dataspectsSystem" MentionsDomainTerm::dataspects workload script Setup "dataspectsSystem" CanBeDeveloped::Yes CanBeInstalled::From https://github.com/dataspects/dataspectsSystem.git CanBeInstalled::Manually HasQualityAssuredBy::Team "DataspectsNonLinuxTeam" MentionsDomainTerm::Environment Variables MentionsDomainTerm::Backing Services MentionsDomainTerm::Site Packages MentionsDomainTerm::Statelessness MentionsDomainTerm::Vendor Packages MentionsDomainTerm::Disposability MentionsDomainTerm::Dependency Declaration MentionsDomainTerm::Process Formation MentionsDomainTerm::Dependency Isolation MustBeMaintained::Yes Solution Add new Solution "Delete Docker authentication token" There might be an (apparently expired) authentication token in ~/.docker/config.json file, which the daemon then tried to use to pull the image from Dockerhub. Deleting the file resolved the problem. Solution "Download again and check $HASH" Download the Composer installation script again and double check the value of the $HASH variable with echo $HASH. Once the installer is verified, you can continue with the next step. Start Add new Start "AAAA" Anybody can say Anything about Anything Anywhere Start "Introduction to FindAndLearn" When people convey/consume knowledge they express/grasp it by things/actions which are described/related by properties. Start "Introduction to FindAndLearn's (FAL) usage of Semantic MediaWiki (SMW)" For FAL SMW fulfills the following requirements: manage information for which there is no (need for a) specialized system manage domain-wide meta information allow for rapidly prototyping preliminary or short-lived subdomain management applications IsPartOfGreaterStart::Start "Introduction to FindAndLearn" Story Add new Story "FindAndLearn" Story "dataspects Usability Metrics" dataspects Usability Metrics formalizes usability conditions to be met by a knowledge management system in order to maximize its usability for users assuming knowledge management roles and responsibilities. SystemComponent Add new SystemComponent "AWS CLI" SystemComponent "DataspectsMediaWikiFeeder" CanBeDeveloped::Yes HasGitHubRepository::https://github.com/dataspects/DataspectsMediaWikiFeeder SystemComponent "Docker Hub login" SystemComponent "Indexing Setups" SystemComponent "Resource Facets" Resource facets are written in dataspectsScript and specify the set of resources that should be extracted as JSON from a specific ResourceSilo. SystemComponent "Resource Feeders" SystemComponent "Resource Indexer Classes" CanBeDeveloped::Yes SystemComponent "Resource Silos" SystemComponent "Ruby Programming Language" CanBeInstalled::Yes SystemComponent "Semantic MediaWiki" wiki.dataspectsSystem is the platform that glues together knowledge scattered across systems into useful recipes and explanations organized by the principles of https://everypageispageone.com/the-book. CanBeInstalled::Yes HasLastProcessValidationStamp::190218byLex SystemComponent "User GnuPG keys" SystemComponent "dataspects Kubernetes cluster" SystemComponent "dataspects SpaCyServer" SpaCyServer is a microservice that provides NLP/NLU-induced text annotations. CanBeDeveloped::Yes SystemComponent "dataspectsSystem Duplicity backup" SystemComponent "datastores" SystemComponent "search.dataspectsSystem" CanBeMonitored::Yes SystemComponent "ui.dataspectsSystem" CanBeBuilt::Yes CanBeDeployed::Yes CanBeDeveloped::Yes CanBeInstalled::Yes CanBeTested::Yes CanBeUpgraded::Yes HasGitHubRepository::https://github.com/dataspects/dataspects-ui Team Add new Team "DataspectsLinuxTeam" Team "DataspectsNonLinuxTeam" Tool Add new Tool "Extension:RDFIO" Tool "draw.io diagrams.net" Tool "git-crypt" https://www.agwa.name/projects/git-crypt/ ds12__isInstalledByCommand::sudo apt install git-crypt Tool "logseq" Untyped Add new Untyped "dataspects Software Architecture" CanBeDeveloped::Yes Untyped "dataspects Software Development" CanBeDeveloped::Yes UsabilityCondition Add new UsabilityCondition "Consume knowledge from pertinent MediaWiki entities/articles/subjects/topics" UsabilityCondition "Consume knowledge within search results" UsabilityCondition "Create EPPO entity type instances" UseCase Add new UseCase "Add a link revealing the target's blurb automatically" ImplementsUseCaseGoal::UseCaseGoal "Link to entities" ValidForSystemComponent::SystemComponent "Semantic MediaWiki" UseCase "Compare content among SMWs and repositories" UseCase "Name + Type to Form" ImplementsUseCaseGoal::UseCaseGoal "Add entities" ValidForSystemComponent::SystemComponent "Semantic MediaWiki" UseCase "Red Links to Forms Chooser" ImplementsUseCaseGoal::UseCaseGoal "Add entities" ValidForSystemComponent::SystemComponent "Semantic MediaWiki" UseCaseGoal Add new UseCaseGoal "Add entities" ConcernsAspect::Editing IsCarriedOutByRole::EndUser UseCaseGoal "Link to entities"

These entity types are managed by dedicated Ontology Components.