Netbox: VM Import Plugin #22
-
|
Hello everyone, |
Beta Was this translation helpful? Give feedback.
Replies: 13 comments 39 replies
-
|
Can you show what you have tried, so I can, may help you with the Problem. The Plugins I build for Import, never did filter something based on Rules. If you can read the VM Attributes from the Netbox API, and you have them as a key:value pair, ` host_obj.update_host(attributes) host_obj.save() |
Beta Was this translation helpful? Give feedback.
-
|
I dind't had much time to look further into it but my idea was to add a " Get VMs" Class based on get_devices in syncer.py that looks like this: def get_vm(self, syncer_only=False):
"""
Read full list of devices
"""
print(f"{CC.OKGREEN} -- {CC.ENDC}Netbox: "\
f"Read all devices (Filter only CMDB Syncer: {syncer_only})")
url = 'virtualisation/vitual-machines/?limit=1000'
if syncer_only:
url += f"&cf_cmdbsyncer_id={self.config['_id']}"
vm = self.request(url, "GET")
return {x['display']:x for x in vm}and than later just copy and edit the netbox plugin and replace the get_devices class with my and edit the terminal commands to a degree that make sense. As much as i can tell the handling of all other stuff like labes etc. should be the same. i think. |
Beta Was this translation helpful? Give feedback.
-
|
In the latest git, I build in the VM import too out of the box. I have currently no net box to test, would be nice if you can check if I have broken something. Now, after checking the Code, I see what you meant by no Labels. There is currently no Label import from Netbox implemented. If you can provide me an API Response from the virtual-machines and point me in the direction, what you need as labels there, I will add the function asap. I would then just again need you to test, since still no netbox here :) |
Beta Was this translation helpful? Give feedback.
-
|
Sure. i'm grabbing the latest build in a moment. In the meantime, here is a responebody for a single VM per id: I hope this helps. if you need something more dont hesitate to ask! |
Beta Was this translation helpful? Give feedback.
-
|
just tryed it out and got some Error with a bunch of html code with Connection Error: Expecting value: line 5 column 1 (char 4) at the end |
Beta Was this translation helpful? Give feedback.
-
|
I updated to add some Debug Outputs. You can enable them after pulling the new version, like described here: https://cmdbsyncer.readthedocs.io/en/latest/basics/lcl_config/ |
Beta Was this translation helpful? Give feedback.
-
|
here the full Error with debug Details
|
Beta Was this translation helpful? Give feedback.
-
|
Netbox says: Page not found to that Endpoint: https://netbox/api/virtualisation/vitual-machines/?limit=10000 I just copied it from you, maybe typo somewhere? Soon as you have the correct link, I can fix in git :) |
Beta Was this translation helpful? Give feedback.
-
|
/virtualization/virtual-machines/ my bad. |
Beta Was this translation helpful? Give feedback.
-
|
@PhXcell |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, my bad, copied the label thing to devices and did not update the loop. Fixed, hope no more errors from there, tried to catch them good as possible |
Beta Was this translation helpful? Give feedback.
-
|
as far as i can tell it looks fine. i just need to look if i change the Attributes for CheckMK. I will look into in in detail after the weekend. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, as far as i can tell everything works for the new features, but I found something odd and i have some questions. Bug: The 'Debug Config' Tab in the CheckMK section is not working and leads into a 404 not found. Question: is it possible in the newer version to manipulate the value off labels? In your videos, the way you did it was different. As an example, i only have the host names in netbox and i want to add my domain part to it to make it a FQDN or strip the networkmask off my ip adresses because CheckMK can't handle "123.123.123.234/25" Suggestion: A search bar for the Hosts and some kind of functionality to import custom field values. I added myself the needed values to labels[label] = sub_data.get('display','custom_hostname')and label_keys = [
'site',
'cluster',
'device',
'role',
'tenant',
'platform',
'primary_ip4',
'primary_ip6',
'custom_fields',
]but a possibility to add them over the interface would be fantastic but not a crucial implementation. |
Beta Was this translation helpful? Give feedback.
-
|
Can you explain what you did here: |
Beta Was this translation helpful? Give feedback.
-
|
changed it to for device, data in self.get_vms().items():
host_obj = Host.get_host(device)
labels = {}
print(f"\n{CC.HEADER}Process VM: {device}{CC.ENDC}")
for label in label_keys:
if sub_data := data.get(label):
if label == 'status':
labels[label] = sub_data.get('value')
else:
labels[label] = sub_data.get('display')
host_obj.update_host(labels)
do_save = host_obj.update_host(labels)
do_save = host_obj.set_account(account_dict=self.config)
if do_save:
host_obj.save()for my purposes |
Beta Was this translation helpful? Give feedback.
-
|
Nope, it's usually because of missing BASE_PREFIX, which needs to be set to your Page Address (Don't have a better solution yet):
Yes, good Ideas, have already somehting in mind |
Beta Was this translation helpful? Give feedback.
-
|
The quick and dirty solution would be the split, but dispite everyone hating on regex it would be the most versetile approach to it. Idk if value manipulation would make sense in other areas. It could be usefull in the generation of the ansible inventory but i have not looked into it so much because i run a seperate ansible server. Ps.: The old rule system was a tiny bit more intuitive, then now in terms of creating labes and Key Attributes. |
Beta Was this translation helpful? Give feedback.
-
|
The Current GIT can now also overwrite/ rewrite Values: Also, using Ansible from another Server is not a Problem and an Usecase I had ;) |
Beta Was this translation helpful? Give feedback.
-
|
The Split works fine for attributes. but im in a bit of a hassle with the export of the the ip adress because i added the split in the primary_ip4 to ipadress rule. now CheckMk dosn't get the ip adress in the host . |
Beta Was this translation helpful? Give feedback.
-
|
Is the Rule set, to make it a Checkmk Attribute? The Syncer does not make everything automatically; otherwise, stuff will happen you don't want. So you need to say, which Attributes should get Labels (Filter), and in the Export Rules you can define which ones should be Checkmk Attributes. It's even possible to use some more Logic here, there is an Example for ipaddress in the link, and below for management board (where you need key: value pairs) Make sure to commit the Changes after, to reset the Cache. |
Beta Was this translation helpful? Give feedback.
-
Details
i made sure to safe it and even imported all hosts fresh to CheckMK. |
Beta Was this translation helpful? Give feedback.
-
|
Works fine now! |
Beta Was this translation helpful? Give feedback.
-
|
Can you clarify what you want to do, don't get it :) ?
|
Beta Was this translation helpful? Give feedback.
-
|
My idea: i have a syncer attribute: role. role has some string of a softwarename and an version number. i want to put a label in Checkmk that looks somthing like this version:fooSoftware12.43.5 long story short.. put a value to a label in CMK |
Beta Was this translation helpful? Give feedback.
-
|
That is just a matter of Whitelist the Attribute. |
Beta Was this translation helpful? Give feedback.
-
|
So far i can do everything i need and it works like a charm. my peronal whishlist for QoL improvments:
Im really looking forward for future updates! |
Beta Was this translation helpful? Give feedback.
-
|
My pleasure then :) Searchbar for hostsDo you know about to option using the Filter: A How to: CMDBSyncer with SSlThat is not a Syncer Topic. If you used the UWSGI Setup, or the Docker Setup together with Reverse Proxy from your Apache or Nginx, the Certificate of the Server is used, same as e.g. Checkmk would do. More placeholder (usefull for the new value edit function)At some Fields, you can use the whole Syntax of Jinja already. Everywhere, the name is something like Template. ldap/azureAD login (not that important)Not yet a priority, hopefully when the system is set you don't need to touch it anymore. But maybe later :) better way to get custom fields over netbox api.Yes, that I have on the list, a way to set the Attributes you want to query as labels. |
Beta Was this translation helpful? Give feedback.
-
|
More placeholder (usefull for the new value edit function) At some Fields, you can use the whole Syntax of Jinja already. Everywhere, the name is something like Template. The first thing that comes to my mind is hostname manipulation for import or export. I somewhere mentioned that i have in my netbox only the hostnames as names. With value manipulation i could do somthing on the line of {{hostname}}.mydomain.foo to make it a FQDN. |
Beta Was this translation helpful? Give feedback.
-
|
Is there a way to debug cronjobs? set up a 3 cronjobs from 0-24 and non of them is triggering. |
Beta Was this translation helpful? Give feedback.
-
|
What would you suggest if i'm runnig the Docker Setup? I tryed to edit the Dockerfile to implement the cronjob into cron. I got it somewhat to Work but it seems not like a long term solution. |
Beta Was this translation helpful? Give feedback.
-
|
I currently have no experience running a crontab inside an application container, I think that is not best practice, but would be happy to be wrong and add your solution permanent. What I do until now: "docker exec CONTAINER ./cmdbsyncer cron run_jobs" |
Beta Was this translation helpful? Give feedback.
-
|
I tried this approach: https://gist.github.com/andyshinn/3ae01fa13cb64c9d36e7 OT: currently looking into a way to send a api request to Ansible Tower/AWX or semaphore to start a playbook after the export to checkmk finished. |
Beta Was this translation helpful? Give feedback.
-
|
One request per host updated, or generally one request after job? |
Beta Was this translation helpful? Give feedback.
-
|
pref. one request after job. I think it is the best way if i restrict ansible to do only x hosts in parallel in the playbook. |
Beta Was this translation helpful? Give feedback.
-
|
Long time no response, sorry :) But when you run the Ansible manually anyway, why don't run a second playbook after? |
Beta Was this translation helpful? Give feedback.
-
|
Welcome back! This is my doing for now for time reasons, but my dream Pipeline would be: Add VM in Netbox -> sync to Syncer -> Sync to CMK -> (Here begins my ToDo/Wishlist) -> query the new added host from CMK -> give hostname to Ansible Tower/AWX/Semaphore API -> Play my default playbook for agent installation. I thougt about doing my own "dumb" Syncer. Just a Script that compares all of my CMK hosts and detects the newest addition. This is in my mind the most easy solution to the problem that the syncer dosen't know if or when my host registerd himselft CMK. |
Beta Was this translation helpful? Give feedback.
-
|
Besides the passing the Data to Ansible Tower, everything works already. About query the Host from Checkmk; therefore you have the inventorize function. It Checks the Hosts in Checkmk, and adds attributes to it, which can be used in the rules again. There you see, for example, if the Agent has a Problem, is not registered or "just" TLS is missing. You can import a sample set of rules to the Ansible part, where you can see some example. It's in the rule_examples folder, and can be imported with the new rule import/export function (Not yet even documented). But the invenorize you find here in the beginning: Sorry, the Documentation could need an Update with all these features :) |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the input. here is the call needed: https://docs.ansible.com/ansible-tower/latest/html/towerapi/api_ref.html#/Job_Templates/Job_Templates_job_templates_launch_create . I completly missed the fact you could get all hosts with "problems" that way. |
Beta Was this translation helpful? Give feedback.
-
|
@PhXcell I just pushed a change regarding your wish: Now the Importer tries to get all the fields and custom fields. Can you check if that's working for you? |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for my late response. Works like a Charm! |
Beta Was this translation helpful? Give feedback.
-
|
@PhXcell And another one for you :) You now have Jinja Powers for the Attributes Rewrites. You can use all the other Attributes in the Template as well to create new values. All Documentation here: https://cmdbsyncer.readthedocs.io/en/latest/basics/rewrite_attributes/ Currently only tested locally on my setup. |
Beta Was this translation helpful? Give feedback.
-
|
Putting the new feature to good use scrapping some internal netbox datajunk off that is only needed for netbox and not having any correlation with CMK. |
Beta Was this translation helpful? Give feedback.
That is just a matter of Whitelist the Attribute.
In the Checkmk Section you have "Filters", there you can Whitelist as Outcome. Set role there, check with
./cmdbsyncer checkmk debug_host YOURHOST, if fine Commit the Changes and export to checkmk