Page History

Scroll Ignore

scroll-pdf	true
scroll-office	true
scroll-chm	true
scroll-docbook	true
scroll-eclipsehelp	true
scroll-epub	true
scroll-html	true

Panel

title	Sommaire

Table of Contents

style	none

Module chargés dans le Broker

Code Block
[2019-09-17 16:49:59] INFO : [ broker-master ] I correctly loaded the modules: [sla,Simple-log,WebUI,Livestatus,Graphite-Perfdata]

Initialisation du module SLA - CHAPITRE [ INITIALISATION ]

Sur le Broker

Création du module

Code Block

[2021-04-13 15:03:08] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] =============          Starting module initialisation     ==============
[2021-04-13 15:07:59] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] Creating new migration process named : broker [ - Module: sla - migration]
[2021-04-13 15:07:59] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] Creating new archive process named : broker [ - Module: sla - archive]
[2021-04-13 15:08:02] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] Creating a queue manager process for the module.
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] Reading module configuration
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] Creating 1 workers

Paramètre de connexion à la base mongo

Code Block

[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ] Parameter load for database connection
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - database ------------------------- :[shinken]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - uri ------------------------------ :[mongodb://localhost/?w=1&fsync=false]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - replica_set ---------------------- :[]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - use_ssh_tunnel ------------------- :[False]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - ssh_keyfile ---------------------- :[~shinken/.ssh/id_rsa]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - ssh_user ------------------------- :[root]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - ssh_tunnel_timeout --------------- :[2]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - use_ssh_retry_failure ------------ :[1]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - auto_reconnect_max_try ----------- :[3]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - auto_reconnect_sleep_between_try - :[3]
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://localhost/?w=1&fsync=false:shinken
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 8.69ms
[2021-04-13 15:08:04] INFO   : [ broker-master   ] [ sla              ] [ INITIALISATION ] [ MONGO ] Ensure mongo index done in 5.01ms

Fin de l'initialisation du module

Code Block

[2021-04-13 15:11:49] INFO : [ broker-master ] [ sla ] [ INITIALISATION ] Load from collection 28 elements info in cache done in 0.84ms
[2021-04-13 15:11:49] INFO : [ broker-master ] [ sla ] [ INITIALISATION ] Found first element monitoring at 17-06-2020 10:42:52
[2021-04-13 15:11:49] INFO : [ broker-master ] [ sla ] [ INITIALISATION ] ============= Module initialized in 234.97ms ==============

Démarrage du processus de migration - chapitre "[ MIGRATION "

Code Block

[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] Sub process ready to work.
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] Migration process broker [ - Module: sla-writer - migration] initialization
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] Parameter load for migrating
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ]    - daily_clean_batch_size ----------------- :[10000]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ]    - daily_clean_pause_time ----------------- :[2.0]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ]    - broker_module_sla_migration_batch_size - :[1000]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ]    - broker_module_sla_migration_pause_time - :[0.1]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ]    - nb_stored_days ------------------------- :[INFINITE]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ]    - time_when_delete_old_SLA --------------- :[03:02]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://localhost/?w=1&fsync=false:shinken
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 6.62ms
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Ensure mongo index done in 4.18ms
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] Load from collection 28 elements info in cache done in 0.92ms
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ migrationMIGRATION (pid:31197) ] [ INITIALISATION ] Migration process broker [ - Module: sla-writer - migration] initialized

Erreurs - La connexion au serveur Mongo n'est pas établie

Avec Tunnel SSH

Code Block

[2021-02-10 10:39:45] ERROR : [ WebUI           ] [ sla              ] [ INITIALISATION ] Initialisation Module: Error : [ WebUI -sla ] [ SSH TUNNEL ] [ MONGODB ] - mongo connection failure : localhost:43577 ==(ssh tunnel)==> 192.168.1.87:22 ==(mongodb)==> 192.168.1.87:27017.

Sans Tunnel SSH

Code Block
[YYYY-MM-DD HH:MM:SS] ERROR : [ WebUI ] [ sla ] [ INITIALISATION ] Initialisation Module: Error : [ broker-master -sla ] [ MONGODB ] - mongo connection failure to 192.168.1.87:27017

Démarrage du processus d'archive - CHAPITRE "[ ARCHIVE "

Code Block

[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Sub process ready to work.
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Starting initialization of archive process [broker [ - Module: sla-writer - archive]]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Reading configuration for sla archive writing
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - time_before_shinken_inactive ----------------------------------:[30]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - minimal_time_before_an_element_become_missing_data ------------:[0]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - minimal_time_before_an_element_become_missing_data_at_startup -:[0]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Parameter load for sla writing
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - size_chunk_to_archive --- :[10000]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - time_between_two_chunks - :[0.1]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - keep_raw_sla_day -------- :[7]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://localhost/?w=1&fsync=false:shinken
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 16.50ms
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Ensure mongo index done in 6.95ms
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Load from collection 28 elements info in cache done in 0.79ms
[2021-04-13 15:40:56] INFO   : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Archive process broker [ - Module: sla-writer - archive] initialized in 51.06ms

Démarrage des workers - CHAPITRE "[ WORKER"

Code Block

[2021-04-13 15:11:50] INFO   : [ broker-master   ] [ sla              ] [ WORKER MANAGER ] Starting worker 0
[2021-04-13 15:11:50] INFO   : [ broker-master   ] [ sla              ] [ WORKER MANAGER ] The worker 0 is now started as pid:29585

Code Block

[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] Sub process ready to work.
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] Staring new worker.
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://localhost/?w=1&fsync=false:shinken
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 5.85ms
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Ensure mongo index done in 15.31ms
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] Load previous state of 28 elements done in 466.86ms.
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] Parameter load for build raw sla
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ]    - time_before_shinken_inactive - :[30]
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ]    - store_output ----------------- :[True]
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ]    - store_long_output ------------ :[True]
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ]    - list_of_stored_output_status - :[set([])]
[2021-04-13 15:40:57] INFO   : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:31252) ] [ INITIALISATION ] New worker start in 468.01ms.

Fonctionnement du module - chapitre [ RUNNING ]

Cas de 2 modules qui archive le même jour en base

Code Block

[2019-09-18 00:08:45] ERROR  : [ broker-master   ] [ sla              ] [ ARCHIVE  (pid:1434)    ] Fail to archive collection 150_2020 (for date 29-05-2020) because it was lock by [broker-int-13 sla2 archive (pid:23460)] and i am [broker-int-13 sla2 archive (pid:23058)] (an other process try to archive this day)

Dans le cas où 2 modules SLA écrivent dans la même base, les 2 modules vont tenter de faire l'archive du jour en même temps. C'est pourquoi nous avons mis un lock afin d'éviter les doublons dans les entrées SLA.

Si vous avez ce log, vérifiez votre configuration si vous n’avez pas 2 modules SLA ou 2 brokers qui écrivent dans la même base Mongo.

Ecriture des états futurs ( les états qu'on reporte sur les jours d'après )

Lors de l'écriture des états futurs, on a le log suivant :

Code Block

[2019-09-18 00:08:45] ERROR  : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:13266) ] Save future states in [31.00ms]. Found [10] entries in base in [0.00ms]. Saving [4] new entries in [31.00ms]. Update [0] entries in [0.00ms]

Une fois par jour, le module nettoie aussi les états futurs à supprimer

Code Block

[2019-09-18 00:08:45] ERROR  : [ broker-master   ] [ sla              ] [ Worker: 0 (pid:13266) ] The clean of future states found [4] entries to delete. There was deleted in [35.00ms (2.00ms for found entries, 33.00ms for delete entries)].

Log de migration des données SLA

A chaque démarrage du module, le module fait une recherche des données à migrer :

Code Block
[2020-07-22 15:11:39] INFO : [ broker-master ] [ sla ] [ MIGRATION (pid:32233) ] Need to migrate 931144 archives

Puis si il y a des données à migrer on log la progression :

Code Block

[2020-07-22 15:11:39] INFO   : [ broker-master   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 1000/931144 archives
[2020-07-22 15:11:39] INFO   : [ broker-master   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 2000/931144 archives
[2020-07-22 15:11:39] INFO   : [ broker-master   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 3000/931144 archives
[2020-07-22 15:11:40] INFO   : [ broker-master   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 4000/931144 archives

Puis enfin :

Si il y a eu une migration :

Code Block
[2020-07-22 15:18:37] INFO : [ broker-master ] [ sla ] [ MIGRATION (pid:32233) ] Migrate 931144 archive done in:7m 0.14s.

Sinon:

Code Block
[2020-07-22 09:58:47] INFO : [ broker-master ] [ sla ] [ MIGRATION (pid:10878) ] No archive need migration. Check done in:1.09ms.

Page tree

Versions Compared

Old Version 34

New Version 35

Key

Module chargés dans le Broker

Initialisation du module SLA - CHAPITRE [ INITIALISATION ]

Sur le Broker

Création du module

Paramètre de connexion à la base mongo

Fin de l'initialisation du module

Démarrage du processus de migration - chapitre "[ MIGRATION "

Erreurs - La connexion au serveur Mongo n'est pas établie

Avec Tunnel SSH

Sans Tunnel SSH

Démarrage du processus d'archive - CHAPITRE "[ ARCHIVE "

Démarrage des workers - CHAPITRE "[ WORKER"

Fonctionnement du module - chapitre [ RUNNING ]

Cas de 2 modules qui archive le même jour en base

Ecriture des états futurs ( les états qu'on reporte sur les jours d'après )

Log de migration des données SLA