View Source

Module chargés dans le Broker

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME    ] I correctly loaded the modules: [sla,Simple-log,WebUI,Livestatus,Graphite-Perfdata]

Initialisation du module SLA - CHAPITRE [ INITIALISATION ]

Sur le Broker

Création du module

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] =============          Starting module initialisation     ==============
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] Creating new migration process named : BROKER_NAME [ - Module: sla - migration]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] Creating new archive process named : BROKER_NAME [ - Module: sla - archive]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] Creating a queue manager process for the module.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] Reading module configuration
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] Creating 1 workers

Paramètre de connexion à la base mongo

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ] MongoDB parameters
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - database__name ————————————————————————————————————————————————— :〖 shinken 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - database__uri —————————————————————————————————————————————————— :〖 mongodb://192.168.1.69/?w=1&fsync=false 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - replica_set ———————————————————————————————————————————————————— :〖  〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - use_ssh_tunnel ————————————————————————————————————————————————— :〖 0 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - ssh_user ——————————————————————————————————————————————————————— :〖 shinken 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - ssh_keyfile ———————————————————————————————————————————————————— :〖 ~shinken/.ssh/id_rsa 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - ssh_tunnel_timeout ————————————————————————————————————————————— :〖 10 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ] AutoReconnect Management
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - database__retry_connection_X_times_before_considering_an_error — :〖 4 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ]    - database__wait_X_seconds_before_reconnect —————————————————————— :〖 3 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://192.168.1.69/?w=1&fsync=false:shinken
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 3.13ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] [ MONGO ] Ensure that the index is present in the database ( and create it if necessary ). Done in 80.84ms

Fin de l'initialisation du module

[YYYY-MM-DD HH:MM:SS] INFO : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] Load from collection 28 elements info in cache done in 0.84ms
[YYYY-MM-DD HH:MM:SS] INFO : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] Found first element monitoring at 17-06-2020 10:42:52
[YYYY-MM-DD HH:MM:SS] INFO : [ BROKER_NAME ] [ sla              ] [ INITIALISATION ] ============= Module initialized in 234.97ms ==============

Démarrage du processus de migration - chapitre "[ MIGRATION ]"

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] Sub process ready to work.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] Migration process BROKER_NAME [ - Module: sla-writer - migration] initialization 
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] Parameter load for migrating
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ]    - daily_clean_batch_size ————————————————— :〖 10000 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ]    - daily_clean_pause_time ————————————————— :〖 2 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ]    - broker_module_sla_migration_batch_size — :〖 1000 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ]    - broker_module_sla_migration_pause_time — :〖 0 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ]    - nb_stored_days ————————————————————————— :〖 -1 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ]    - time_when_delete_old_SLA ——————————————— :〖 03:02 〗 
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://localhost/?w=1&fsync=false:shinken
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 6.62ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] [ MONGO ] Ensure mongo index done in 4.18ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] Load from collection 28 elements info in cache done in 0.92ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:31197) ] [ INITIALISATION ] Migration process BROKER_NAME [ - Module: sla-writer - migration] initialized

Erreurs - La connexion au serveur Mongo n'est pas établie

Avec Tunnel SSH

[YYYY-MM-DD HH:MM:SS] ERROR : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] Initialisation Module: Error : [ BROKER_NAME -sla ] [ SSH TUNNEL ] [ MONGODB          ]  - mongo connection failure : localhost:43577 ==(ssh tunnel)==> 192.168.1.87:22 ==(mongodb)==> 192.168.1.87:27017.

Sans Tunnel SSH

[YYYY-MM-DD HH:MM:SS] ERROR : [ BROKER_NAME   ] [ sla              ] [ INITIALISATION ] Initialisation Module: Error : [ BROKER_NAME  -sla ] [ MONGODB          ]   - mongo connection failure to 192.168.1.87:27017

Démarrage du processus d'archive - CHAPITRE "[ ARCHIVE ] "

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Sub process ready to work.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Starting initialization of archive process [BROKER_NAME [ - Module: sla-writer - archive]]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Reading configuration for sla archive writing 
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - time_before_shinken_inactive —————————————————————————————————— :〖 30 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - minimal_time_before_an_element_become_missing_data ———————————— :〖 60 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - minimal_time_before_an_element_become_missing_data_at_startup — :〖 600 〗 
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Parameter load for sla writing 
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - size_chunk_to_archive ——— :〖 10000 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - time_between_two_chunks — :〖 0,1 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ]    - keep_raw_sla_day ———————— :〖 7 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://localhost/?w=1&fsync=false:shinken
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 16.50ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] [ MONGO ] Ensure that the index is present in the database ( and create it if necessary ). Done in 5.01ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Load from collection 28 elements info in cache done in 0.79ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:31201) ] [ INITIALISATION ] Archive process BROKER_NAME [ - Module: sla-writer - archive] initialized in 51.06ms

Démarrage des workers - CHAPITRE "[ WORKER]"

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER MANAGER ] Starting worker 0
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER MANAGER ] The worker 0 is now started as pid:29585

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] Sub process ready to work.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] Staring new worker.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Creating connection to sla database [shinken]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Try to open a Mongodb connection to mongodb://localhost/?w=1&fsync=false:shinken
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Mongo connection established in 5.85ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] [ MONGO ] Ensure that the index is present in the database ( and create it if necessary ). Done in 5.01ms
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] Load previous state of 28 elements done in 466.86ms.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] Parameter load for build raw sla 
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ]    - store_output ——————————————————————————————————————— :〖 1 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ]    - store_long_output —————————————————————————————————— :〖 1 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ]    - list_of_stored_output_status ——————————————————————— :〖 set([]) 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ]    - minimal_time_before_an_element_become_missing_data — :〖 60 〗
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ]    - time_before_shinken_inactive ——————————————————————— :〖 30 〗 
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] [ INITIALISATION ] New worker start in 468.01ms.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:31252) ] Now running..

Arrêt du module

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ MODULES-MANAGER  ] [ SHUTDOWN ] Stopping module sla
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ MODULES-MANAGER  ] [ SHUTDOWN ] [ sla ] [ module-type=sla ] Start stopping all process of SLA Module Broker
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] Stopping process shinken-BROKER_NAME [ - Module: sla - migration ]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] The [shinken-BROKER_NAME [ - Module: sla - migration ]] sub-process is still alive, I help it to die
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] The [shinken-BROKER_NAME [ - Module: sla - migration ]] pid:[13465] sub-process is still alive, I kill it (kill -9)
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] Stopping process shinken-BROKER_NAME [ - Module: sla - archive ]
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] The [shinken-BROKER_NAME [ - Module: sla - archive ]] sub-process is still alive, I help it to die
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] The [shinken-BROKER_NAME [ - Module: sla - archive ]] pid:[13467] sub-process is still alive, I kill it (kill -9)
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] Stopping all workers.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] Stopping module process pid=13512
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ WORKER: 0 (pid:13512) ] exiting now..
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] Trying to stop module
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] The queue manager process is now stopped.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] The queue manager process is now clean.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] Stopping module was a success.
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ MODULES-MANAGER  ] [ SHUTDOWN ] [ sla ] [ module-type=sla ] Stopping all process of SLA Module Broker done

Fonctionnement du module - chapitre [ SLA COLLECTIONS STATS ]

Ce chapitre traite des données globales de la base de données concernant le module SLA ( comme le nombre d'éléments uniques dans la base archive ).

Chapitre [ UNIQUE ELEMENTS IN ARCHIVE ]

Ce chapitre traite du maintien ( récupération, lecture, mise à jour ) d'un compteur représentant le nombre d'éléments total archivés.

Chargement des données en mémoire

Pour un accès plus rapide ( par les checks par exemple ) la valeur du compteur est stockée en mémoire et nous sommes avertis lorsque le module essaye d'aller chercher la précédente valeur en base.

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the previous count of unique elements in archive from "sla_collections_stats" collection: Try 1/3

Aucune donnée de trouvée

Lorsque la collection Mongo contenant le compteur n'existe pas, un message nous avertis pour nous prévenir qu'elle va être créée

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] No count of unique elements in archive found in "sla_collections_stats" collection
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Creating the database document for archived elements count

Des données ont été trouvées

Si le compte est présent en base, alors nous sommes informés de la quantité d'éléments et de l'ancienneté de la donnée

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla             ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Found [ X ] unique elements in archive in "sla_collections_stats" collection, dating from [ YYYY-MM-DD HH:MM:SS ]

Rafraichissement du compteur

Si lors du chargement les données sont considérées comme trop vielles ( plus d'un jour ) alors elles seront rafraichies depuis la base d'archive.

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] The count of unique elements in archive needs to be refreshed because it is older than one day

Le compteur peut être rafraichi sous ces conditions :

Premier calcul
Il est plus vieux qu’un jour
Le module d'archivage ou de migration a changé le contenu des archives

En cas d'erreur, la récupération des données va être relancée jusqu'à trois fois.

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the count of unique elements in archive from archive database: Try 1/3

Quand la récupération du nombre d'éléments dans la base d'archive réussi, alors nous sommes informés du nombre en question

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Count of unique elements in archive found: [ X ]

Les cas d'erreur

La récupération de la précédente valeur du compteur va s'effectuer au maximum trois fois. Pour chaque essai qui échoue, l'erreur en question est indiquée en avertissement, sauf lors du dernier essai où elle sera montrée en erreur.

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME    ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the previous count of unique elements in archive from "sla_collections_stats" collection: Try 1/3
[YYYY-MM-DD HH:MM:SS] WARNING: [ BROKER_NAME    ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the previous count of unique elements in archive from "sla_collections_stats" collection failed because of this error: AutoReconnect
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME    ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the previous count of unique elements in archive from "sla_collections_stats" collection: Try 2/3
[YYYY-MM-DD HH:MM:SS] WARNING: [ BROKER_NAME    ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the previous count of unique elements in archive from "sla_collections_stats" collection failed because of this error: AutoReconnect
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME    ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the previous count of unique elements in archive from "sla_collections_stats" collection: Try 3/3
[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME    ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the previous count of unique elements in archive from "sla_collections_stats" collection failed because of this error: AutoReconnect

La récupération du compteur depuis la base d'archive va s'effectuer au maximum trois fois. Pour chaque essai qui échoue, l'erreur en question est indiquée en avertissement, sauf lors du dernier essai où elle sera montrée en erreur.

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the count of unique elements in archive from archive database: Try 1/3
[YYYY-MM-DD HH:MM:SS] WARNING: [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the count of unique elements in archive from archive database failed because of this error: AutoReconnect
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the count of unique elements in archive from archive database: Try 2/3
[YYYY-MM-DD HH:MM:SS] WARNING: [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the count of unique elements in archive from archive database failed because of this error: AutoReconnect
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the count of unique elements in archive from archive database: Try 3/3
[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME     ] [ sla              ] [ SLA COLLECTIONS STATS ] [ UNIQUE ELEMENTS IN ARCHIVE ] Getting the count of unique elements in archive from archive database failed because of this error: AutoReconnect

Fonctionnement du module - chapitre [ ARCHIVING ]

Cas de 2 modules qui archive le même jour en base

[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME   ] [ sla              ] [ ARCHIVE  (pid:1434)    ] [ ARCHIVING ] Fail to archive collection 150_2020 (for date 29-05-2020) because it was lock by [broker-int-13 sla2 archive (pid:23460)] and i am [broker-int-13 sla2 archive (pid:23058)] (an other process try to archive this day)

Dans le cas où 2 modules SLA écrivent dans la même base, les 2 modules vont tenter de faire l'archive du jour en même temps. C'est pourquoi nous avons mis un lock afin d'éviter les doublons dans les entrées SLA.

Si vous avez ce log, vérifiez votre configuration si vous n’avez pas 2 modules SLA ou 2 brokers qui écrivent dans la même base Mongo.

Fonctionnement du module - chapitre [ POSTPONED STATE ]

Ce composant gère les états qui durent jusqu'au lendemain (*états reportés*)

Chargement des états reportés

Au démarrage du module, ou lors d'un rechargement de configuration, le composant des états reportés charge sa rétention en base de données.

Le log suivant détaille l'opération, et le temps passé.

[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME   ] [ sla              ] [ WORKER: N (pid:PID) ] [ POSTPONED STATE ] Postponed states loading found X entries in Y.YYms [ including database fetching in A.AAms and data building in B.BBms ]

[2021-12-09 12:09:40] INFO   : [ broker-master   ] [ sla              ] [ WORKER: 0 (pid:2573) ] [ POSTPONED STATE ] Postponed states loading found 6 entries in 0.41ms [ including database fetching in 0.39ms and data building in 0.02ms ]

Sauvegarde des états reportés

Périodiquement, quand le module reçoit des états qui durent jusqu'au lendemain, le composant va les enregistrer en base de données pour assurer leur persistance en cas de redémarrage

Le log suivant détaille l'opération et le temps qu'elle a pris.

[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME   ] [ sla              ] [ WORKER: N (pid:PID) ] [ POSTPONED STATE ] Saved postponed states in A.AAms. Local cache lock acquisition in B.BBms, cache copy in C.CCms. Fetched P entries from base in D.DDms. Saved Q new entries in base in E.EEms. Updated R entries in base in F.FFms. Removed S entries from cache in G.GGms [ including lock acquisition: H.HHms, deletion: I.IIms ]

[2021-12-09 12:09:42] INFO   : [ broker-master   ] [ sla              ] [ WORKER: 0 (pid:2573) ] [ POSTPONED STATE ] Saved postponed states in 2.81ms. Local cache lock acquisition in 0.02ms, cache copy in 0.03ms. Fetched 6 entries from base in 1.11ms. Saved 0 new entries in base in 0.00ms. Updated 6 entries in base in 1.63ms. Removed 6 entries from cache in 0.01ms [ including lock acquisition: 0.01ms, deletion: 0.01ms ]

Nettoyage des états reportés

Périodiquement, le composant va rechercher et supprimer les entrées obsolètes relatives aux états reportés qui peuvent subsister en base de données.

Le module n'a pas reçu sa configuration

Si l'opération de nettoyage est demandée alors que le module n'a pas encore reçu la liste des éléments qu'il gère, l'opération de nettoyage est annulée.

Le log suivant détaille le temps passer pour faire cette vérification.

[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME   ] [ sla              ] [ WORKER: N (pid:PID) ] [ POSTPONED STATE ] The clean of postponed states found no entry to delete as inventory is waiting to be filled up. The search took A.AAms.

[2021-12-09 12:09:42] INFO   : [ broker-master   ] [ sla              ] [ WORKER: 0 (pid:2573) ] [ POSTPONED STATE ] The clean of postponed states found no entry to delete as inventory is waiting to be filled up. The search took 0.01ms.

Pas d'éléments à nettoyer

Si le composant n'a trouvé aucun élément à retirer de la rétention en base de données, le log suivant indique le temps passé pour arriver à cette conclusion.

[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME   ] [ sla              ] [ WORKER: N (pid:PID) ] [ POSTPONED STATE ] The clean of postponed states found no entry to delete. The search took A.AAms [ including inventory fetching: B.BBms, cache lock waiting: C.CCms, cache copying: D.DDms, cache formatting: E.EEms, removed states computing: F.FFms , database matching: G.GGms including [ data fetch: H.HHms, compute: I.IIms ] ].

[2021-12-09 12:09:51] INFO   : [ broker-master   ] [ sla              ] [ WORKER: 0 (pid:2573) ] [ POSTPONED STATE ] The clean of postponed states found no entry to delete. The search took 5.67ms [ including inventory fetching: 0.03ms, cache lock waiting: 0.00ms, cache copying: 0.02ms, cache formatting: 0.00ms, removed states computing: 0.01ms , database matching: 5.61ms including [ data fetch: 5.55ms, compute: 0.00ms ] ].

Des éléments ont été supprimés

Quand l'opération de nettoyage des états reportés obsolètes trouve des éléments à supprimer, le log suivant détaille le temps passé sur chaque étape, de la recherche à la suppression des éléments.

[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME   ] [ sla              ] [ WORKER: N (pid:PID) ] [ POSTPONED STATE ] The clean of postponed states removed N entries and took A.AAms. The search took %s [ including inventory fetching: B.BBms, cache lock waiting: C.CCms, cache copying: D.DDms, cache formatting: E.EEms, removed states computing: F.FFms , database matching: G.GGms including [ data fetch: H.HHms, compute: I.IIms ] ]. Database remove took J.JJms.

[2021-12-09 12:09:51] INFO   : [ broker-master   ] [ sla              ] [ WORKER: 0 (pid:2573) ] [ POSTPONED STATE ] The clean of postponed states removed 6 entries and took 6.67ms. The search took 5.67ms [ including inventory fetching: 0.03ms, cache lock waiting: 0.00ms, cache copying: 0.02ms, cache formatting: 0.00ms, removed states computing: 0.01ms , database matching: 5.61ms including [ data fetch: 5.55ms, compute: 0.00ms ] ]. Database remove took 1.00ms.

Fonctionnement du module - chapitre [ RUNNING ]

Ecriture des états futurs ( les états qu'on reporte sur les jours d'après )

Lors de l'écriture des états futurs, on a le log suivant :

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ WORKER: 0 (pid:10277) ] [ RUNNING ] Save future states in [2.88ms]. Found [4] entries in base in [1.42ms]. Saving [1] new entries in [1.34ms]. Update [0] entries in [0.04ms]

Une fois par jour, le module nettoie aussi les états futurs à supprimer

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME ] [ sla              ] [ WORKER: 0 (pid:10277) ] [ RUNNING ] The clean of future states found [1] entries to delete. There was deleted in [2.28ms (1.28ms for found entries, 1.03ms for delete entries)].

Log de migration des données SLA

A chaque démarrage du module, le module fait une recherche des données à migrer :

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:32233) ] Need to migrate 931144 archives

Puis si il y a des données à migrer on log la progression :

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 1000/931144 archives
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 2000/931144 archives
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 3000/931144 archives
[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrating in progress 4000/931144 archives

Puis enfin :

Si il y a eu une migration :

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:32233) ] Migrate 931144 archive done in:7m 0.14s.

Sinon:

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MIGRATION (pid:10878) ] No archive need migration. Check done in:1.09ms.

Fonctionnement du module - chapitre [ MONGO ]

Log de gestion des erreurs

Lorsqu'une erreur survient sur la connexion vers mongodb, Shinken tente de refaire une connexion à la base.

Voir Configuration du Module SLA

Tentative en cours :

[YYYY-MM-DD HH:MM:SS] INFO   : [ BROKER_NAME   ] [ sla              ] [ MONGO ] Mongo raised ( ERROR TEXT ) on the operation  OPERATION_NAME on COLLECTION_NAME. Operation failed : X/Y

Toutes les tentatives on échouées :

[YYYY-MM-DD HH:MM:SS] ERROR  : [ BROKER_NAME   ] [ sla              ] [ MONGO ] Mongo raised ( ERROR TEXT ) on the operation  OPERATION_NAME on COLLECTION_NAME. Operation failed : Y/Y. We tried Y time but it kept failing.

X : est le nombre de fois où l'opération a été tenté
Y : est le nombre de tentative maximal de l'opération
OPERATION_NAME : nom de l'opération. Exemple : find, find_one, list_name_collections, save ...
COLLECTION_NAME : nom de la collection MongoDB