NOC comes with large amount of bundled MIBs, stored in share/mibs folder. MIBs are used by Fault Management to:
- Convert OIDs to symbolical names (i.e. 22.214.171.124.126.96.36.199 -> RFC1213-MIB::sysDescr), which, in order, used in classification rules
- Convert varbind value to symbolical name, using value type and syntax (like decoding binary MAC addresses, converting enumerations from integers to strings and so on)
NOC never uses MIBs directly, it stores compiled MIB representation in database instead. MIBs are stored in noc.mibs and noc.mibdata collections in mongodb. ./noc sync-mibs (usually called from ./scripts/post-update) does all the work of detection which MIBs need to be uploaded and uploading them to the database. The obvious way has several drawbacks:
MIBs are rarely used alone. Common MIB uses IMPORT statements to get macroses and definitions from the other MIBs. As a result, sync-mibs sometimes used to load and parse tens of MIBs to parse a single one.
As a consequence, loading a bunch of MIBs, sync-mibs rereads and parses same MIBs again and again
- sync-mibs needs a time to resolve MIB dependencies to load them in proper order
- sync-mibs uses libsmi as backend to parse mib, which adds an additional dependency
- MIBs are plagued by errors, causing smidump to dump all the errors, severe polluting an output
As a result, two direct consequences follow:
sync-mibs is rather slow, when loading all the MIBs for the first time (during installation), though works much faster during an updates
Installation/update logs are filled by insignificant MIB-related details
NOC-153 - Compiled MIBs in the distribution - RESOLVED addresses those two drawbacks in simple and elegant way: once loaded in the database, MIBs can be serialized and bundled with distribution along with plain-text MIBs. Serialized MIBs stored in the gzip-compressed JSON form into fm/collections/mibs folder. From now on, sync- mibs never deals with plain-text MIBs, its only uploads missed or updated MIBs from distribution bundle to database. Note the custom MIBs processing order remains unchanged.
Quick tests shows, it takes about 89 seconds to parse and upload all MIBs to database on my MacBook Air, while it takes about 15 seconds to unroll all serialized MIBs.
Simple tutorial to bundle new MIB to distribution (for commiters only):
Import MIB into database:
Serialize MIB to temporary directory:
Note: the bundled file name must exactly match MIB name in definition
Move plain-text MIB from local/ to distribution:
Move compiled MIB from temporary directory to distribution:
Note: If when you type command ./noc sync-mibs you saw like that: