OPNsense Forum

English Forums => Development and Code Review => Topic started by: skatopn on February 01, 2024, 12:27:44 PM

Title: [RESOLVED]Import list of Aliases from CSV
Post by: skatopn on February 01, 2024, 12:27:44 PM
Hi,

A lot of people - a LOT - use Excel (and similar tools) to manage all sorts of data sets, including but not limited to network data sets, such as lists of hosts, lists of policies (Firewall and NAT), and lists of service objects.

I want to be able to import from CSV files (using appropriate formatting/columns):

Are any of these currently possible?

Creating all these objects manually is a very tedious process, but tools like Excel can save a lot of time when you are developing a complete list of objects that follow pre-defined naming conventions, or when you are creating a complete Ruleset for a new firewall deployment.

The ability to do this (CSV object import) would make migrating from another vendor to OPNsense much less painful.

Feature Request please :)
Title: Re: Import list of Aliases from CSV
Post by: Patrick M. Hausen on February 01, 2024, 12:45:38 PM
Feature requests are best opened as issues on github. This is the community forum - users helping users.
Title: Re: Import list of Aliases from CSV
Post by: franco on February 01, 2024, 05:35:24 PM
There is a JSON based export/input. I don't think we'll be adding CSV if you don't count the ability to paste CSV into the input field which has been working forever.


Cheers,
Franco
Title: Re: Import list of Aliases from CSV
Post by: skatopn on February 02, 2024, 05:28:19 AM
Quote from: Patrick M. Hausen on February 01, 2024, 12:45:38 PM
Feature requests are best opened as issues on github. This is the community forum - users helping users.

Ok, but this sub-topic is "Development and Code Review"...it seemed to me to be a fitting place. Other discussions about functionality are also under here and it read to me like people here are also developers, not just users.

I will see if I can submit it on Github...

Perhaps someone should create a sticky post under "Development and Code Review" and call it "Submitting Feature Requests" and put a link to the github repo and have instructions for submitting feature requests?

What is the link to the repo?
Title: Re: Import list of Aliases from CSV
Post by: skatopn on February 02, 2024, 05:29:39 AM
Quote from: franco on February 01, 2024, 05:35:24 PM
There is a JSON based export/input. I don't think we'll be adding CSV if you don't count the ability to paste CSV into the input field which has been working forever.


Cheers,
Franco

I did not know you could paste a CSV list into it. Is there any documentation or help as to formatting/column requirements for the CSV data?
Title: Re: Import list of Aliases from CSV
Post by: franco on February 02, 2024, 02:00:17 PM
It's not a column format. It's a CSV per field for adding multiple entries in tokenized inputs like the alias content.


Cheers,
Franco
Title: Re: Import list of Aliases from CSV
Post by: skatopn on February 04, 2024, 04:46:16 AM
Quote from: franco on February 02, 2024, 02:00:17 PM
It's not a column format. It's a CSV per field for adding multiple entries in tokenized inputs like the alias content.


Cheers,
Franco

Forgive me, but I don't quite understand your statement.
Would you be kind enough to explain or provide an example, or maybe just point me to an example or tutorial showing how this is done?

I'm afraid the GUI is not intuitive enough. I can see the "Content" field, but when I try to add CSV objects I get the error: "Entry "test_obj1" is not a valid hostname, IP address or range.".

If by "paste CSV" you mean pasting a list of IP addresses or hostnames into the Content field of one object/alias, that is not what I want.

I want to be able to create a whole list of separate object aliases, each with their own IP address or hostname and their own attributes.

Does that make sense?
Title: Re: Import list of Aliases from CSV
Post by: franco on February 04, 2024, 01:06:54 PM
Sure, for this you use the import/export JSON feature. It's the little buttons in the lower right corner of the main table.


Cheers,
Franco
Title: Re: Import list of Aliases from CSV
Post by: skatopn on February 08, 2024, 12:13:17 AM
Thanks.

JSON is not CSV though, so I will have to try and come up with a way to convert CSV to JSON.
Looking some API examples, it appears that I don't need to create an object UUID first, so this might be easier than I first thought.

I will do some testing and report back.
Title: [RESOLVED]Re: Import list of Aliases from CSV
Post by: skatopn on March 11, 2024, 02:28:16 AM
Quote from: skatopn on February 08, 2024, 12:13:17 AM
Thanks.

JSON is not CSV though, so I will have to try and come up with a way to convert CSV to JSON.
Looking some API examples, it appears that I don't need to create an object UUID first, so this might be easier than I first thought.

I will do some testing and report back.

This is resolved in this post:
https://forum.opnsense.org/index.php?topic=36687.msg179205#msg179205 (https://forum.opnsense.org/index.php?topic=36687.msg179205#msg179205)
Title: Re: [RESOLVED]Import list of Aliases from CSV
Post by: Baldaresan on December 17, 2024, 11:50:08 AM
If you want to import a list of aliases from a CSV into Excel, it's pretty simple. First, you'll want to open your Excel sheet and then click the "Data" tab. From there, hit "Get Data" and select "From File" and then "From CSV." Once you've picked the file, Excel will guide you through a preview, and you can adjust settings, like the delimiter, if needed, to ensure everything imports correctly.If you're unsure about formatting or have issues, there's a great step-by-step guide on MyExcelOnline's blog (https://www.myexcelonline.com/blog/). It's easy to follow and might save you some headaches if you're new to CSV importing. I found their tips super helpful when I got the hang of it.