Informatics (Internal)

This documentation is only visible to GBIF staff.

We intend to document internal procedures, systems and resources, especially those concerning several systems.

System overview

Data centre and servers

Basic services

PostgreSQL

Monitoring

Data cluster

  • Hadoop

    • Sizing, installation and configuration

    • Components:

      • Zookeeper

      • HDFS

      • HBase

      • YARN

      • Hive

      • Spark

    • Maintenance

    • Incident response

  • Stackable

    • Sizing, installation and configuration

    • Components:

      • ToDo

    • Maintenance

    • Incident response

  • ElasticSearch

    • Sizing, installation and configuration

    • Maintenance

    • Incident response

Pipelines

  • Installation overview

  • Configuration overview

  • Maintenance / incident response

    • Identifier problems

    • Failed ingestions

    • Reprocessing all data

    • Recovery/rollback

Web services

  • Registry-ws

  • Checklistbank-ws

  • Occurrence-ws

  • Maps (vectortile-server, mapnik-server)

  • Literature

  • Thumbor

  • etc…​ (TODO)

Other services

Development services

  • Jenkins

  • Sonar

  • Nexus