After update: Squid URL could not be retrieved

[jordan_richardo]

Hello there,

I had a system originally running on 18.1.3 that has been gradually upgraded up to 18.7.9 which went smoothly with exception of the Squid proxy. I'm not sure where it finally died, somewhere between 18.7.5 and 18.7.9 it seems though.

Basically, the service would not start initially and I came to find there was a rogue Squid service running which I killed. After some reboots I now consistently get the squid service up but am unable to get anywhere. It seems that some portion that parses the URL's is not feeding them correctly. I have attached an image so you can see the error as well.

This proxy was originally setup as transparent, with SSL and certificate installed on all computers as well. Everything was working properly originally. I have also tried deleting the squid files, removing the squid section from the config.xml and doing the "Reinstall" option for the squid3 package in various combinations with reboots in there too for good measure

I'm hoping someone can help me track this down, it almost looks as though the squid config doesn't update on the error page. For example, changing the admin email doesn't change the field "webmaster". My gut tells me there is a file somewhere creating an override, but I have no idea where else to look.

Thanks for reading,
Jordan

[guest19757]

Hello there,

The problem with this post, it's missing a lot context to help diagnose the problem.

1. Post a copy of you current squid.conf
2. Post logs, squid logs, general logs, confid logs.


If you remove all your custom configuration and start anew with default settings, does Squid even work at all?

Regards

[jordan_richardo]

Hello there,

Sorry for not including more detail. I have attached the squid.conf from /usr/local/etc/squid, is this the correct one? My understanding was that configuration was parsed from /conf/config.xml, is that not the case?

I have tried removing all settings from /conf/config.xml between the <squid> flags, and deleting all files in /usr/local/etc/squid and rebooting. After which I have done a really basic setup with the local network, HTTP proxy only, and receive the same error message.

I snipped the more current parts of the two logs, there isn't anything overly helpful that I can find unfortunately. Is there a way that I can load the GUI config and still start Squid in a high verbose mode from the CLI?

Squid log:

Code: [Select]

2019/01/01 19:55:50 kid1| storeLateRelease: released 0 objects
2019/01/01 19:55:49| pinger: ICMPv6 socket opened
2019/01/01 19:55:49| pinger: ICMP socket opened.
2019/01/01 19:55:49| pinger: Initialising ICMP pinger ...
2019/01/01 19:55:49 kid1| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 11 flags=9
2019/01/01 19:55:49 kid1| Adaptation support is off.
2019/01/01 19:55:49 kid1| Squid plugin modules loaded: 0
2019/01/01 19:55:49 kid1| Pinger socket opened on FD 13
2019/01/01 19:55:49 kid1| HTCP Disabled.
2019/01/01 19:55:49 kid1| Finished loading MIME types and icons.
2019/01/01 19:55:49 kid1| Set Current Directory to /var/squid/cache
2019/01/01 19:55:49 kid1| Using Least Load store dir selection
2019/01/01 19:55:49 kid1| Max Swap size: 0 KB
2019/01/01 19:55:49 kid1| Max Mem size: 262144 KB
2019/01/01 19:55:49 kid1| Using 8192 Store buckets
2019/01/01 19:55:49 kid1| Target number of buckets: 1008
2019/01/01 19:55:49 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 objects
2019/01/01 19:55:49 kid1| Store logging disabled
2019/01/01 19:55:49 kid1| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2019/01/01 19:55:49 kid1| Logfile Daemon: opening log /var/log/squid/access.log
2019/01/01 19:55:49 kid1| Logfile: opening log daemon:/var/log/squid/access.log
2019/01/01 19:55:49 kid1| Adding nameserver 8.8.4.4 from /etc/resolv.conf
2019/01/01 19:55:49 kid1| Adding nameserver 8.8.8.8 from /etc/resolv.conf
2019/01/01 19:55:49 kid1| Adding nameserver 127.0.0.1 from /etc/resolv.conf
2019/01/01 19:55:49 kid1| Adding domain dawsongray.on.ca from /etc/resolv.conf
2019/01/01 19:55:49 kid1| DNS Socket created at 0.0.0.0, FD 8
2019/01/01 19:55:49 kid1| DNS Socket created at [::], FD 6
2019/01/01 19:55:49 kid1| Initializing IP Cache...
2019/01/01 19:55:49 kid1| With 113967 file descriptors available
2019/01/01 19:55:49 kid1| Process Roles: worker
2019/01/01 19:55:49 kid1| Process ID 32936
2019/01/01 19:55:49 kid1| Service Name: squid
2019/01/01 19:55:49 kid1| Starting Squid Cache version 3.5.28 for amd64-portbld-freebsd11.1...
2019/01/01 19:55:49 kid1| Set Current Directory to /var/squid/cache
2019/01/01 19:55:48| No cache_dir stores are configured.
2019/01/01 19:55:48| Creating missing swap directories
2019/01/01 19:55:48| Set Current Directory to /var/squid/cache

[jordan_richardo]

Configd.log:

Code: [Select]

Jan  1 19:30:40 core configd.py: [7c3c03db-1a59-4a27-a197-77c7e1422dc7] starting proxy
Jan  1 19:30:42 core configd.py: [e071ae0a-6edc-4fb8-b0ce-53343f88b3e1] request proxy status
Jan  1 19:30:42 core configd.py: [ec207a79-6ef8-4f58-acf3-9ea82f59a239] request proxy status
Jan  1 19:31:29 core configd.py: [0fd25d89-17c5-466c-a671-cbca4ee95083] request pfctl byte/packet counters
Jan  1 19:31:36 core configd.py: [963eaaa7-5962-41e6-9a05-81c0482388ad] request pfctl byte/packet counters
Jan  1 19:31:42 core configd.py: [45cc09f7-338d-4d0e-9268-08add0a00b0b] request pfctl byte/packet counters
Jan  1 19:31:48 core configd.py: [bf7ddfb4-61c8-441b-8583-58e08b1c2fc0] request pfctl byte/packet counters
Jan  1 19:31:55 core configd.py: [d00553dc-336e-4436-946d-08c70bd4182a] request pfctl byte/packet counters
Jan  1 19:32:02 core configd.py: [6b476acb-ba0c-4682-9ede-48563ffd99ce] request pfctl byte/packet counters
Jan  1 19:32:08 core configd.py: [96d0fe07-04fc-4f03-9088-1b103eb679ff] request pfctl byte/packet counters
Jan  1 19:32:14 core configd.py: [70554249-d5bc-4d08-8399-f59396a1bbee] request pfctl byte/packet counters
Jan  1 19:32:20 core configd.py: [c6975c70-6c0c-4ae3-bc85-acdcc9fb9e41] request pfctl byte/packet counters
Jan  1 19:32:26 core configd.py: [0f4797a0-ce8e-44fa-a49c-a783768cb2b0] request pfctl byte/packet counters
Jan  1 19:32:32 core configd.py: [1258908d-c311-461f-95a7-a5f948341aef] request pfctl byte/packet counters
Jan  1 19:32:39 core configd.py: [1179e767-e517-4d90-b2d9-10ddccc5abce] request pfctl byte/packet counters
Jan  1 19:32:45 core configd.py: [2564ebaf-abb0-4082-91b8-3b71e6e173c2] request pfctl byte/packet counters
Jan  1 19:32:51 core configd.py: [eb4a585a-068f-412d-8a89-c77ebaf728d7] request pfctl byte/packet counters
Jan  1 19:32:57 core configd.py: [870ee92b-5b2c-4ec5-8ab6-c17cc3a18f16] request pfctl byte/packet counters
Jan  1 19:33:04 core configd.py: [d90cc756-3e59-4870-bf56-083e29bad24c] request pfctl byte/packet counters
Jan  1 19:33:09 core configd.py: [124a4df8-c1ca-4bb3-b7e7-062f770e0cd4] request proxy status
Jan  1 19:35:16 core configd.py: [cafbd5c2-4bcf-4b04-8d8b-ec5a58693d26] stopping proxy
Jan  1 19:35:45 core configd.py: [6cf66287-0039-46b7-aaed-bd9ee984d3e5] restarting proxy
Jan  1 19:35:45 core configd.py: [fe27dc2a-c48e-48d1-8089-8b3b22be7659] generate template OPNsense/Proxy
Jan  1 19:35:45 core configd.py: [d502fd25-87e1-4f31-8170-5173ba2fc691] request proxy status
Jan  1 19:35:45 core configd.py: [d71ca99e-aa69-4a3e-92a4-471eb4edaaf7] request proxy status
Jan  1 19:35:45 core configd.py: [21493d6b-1db0-4731-b32c-593019b665cd] reload proxy
Jan  1 19:35:45 core configd.py: [e8b5aedc-8e9d-4990-b678-176619bc4cbb] request proxy status
Jan  1 19:35:46 core configd.py: [d719f7c1-70a1-4d92-9995-3d6e1d00d84a] request proxy status
Jan  1 19:38:07 core configd.py: [242b25b2-b12d-4a66-a7fc-078a780c6544] request proxy status
Jan  1 19:38:28 core configd.py: [fa627d7f-cc4f-4327-a6f7-e5856628f6ca] stopping proxy
Jan  1 19:38:51 core configd.py: [99f9ea78-c6e0-4967-829a-416f6bf29aad] restarting proxy
Jan  1 19:38:51 core configd.py: [93771d18-cd69-4902-9e9b-636639cc84b3] generate template OPNsense/Proxy
Jan  1 19:38:51 core configd.py: [6cf94aaf-35f5-47ce-8760-52f2feb60789] request proxy status
Jan  1 19:38:52 core configd.py: [47fc9d87-3c63-48d3-a092-c6527d108302] request proxy status
Jan  1 19:38:52 core configd.py: [4f27f7f3-a722-4cb9-b6da-8405398c40d6] reload proxy
Jan  1 19:38:52 core configd.py: [fd66f80e-abde-4ba3-a465-798c77f4edb0] request proxy status
Jan  1 19:38:52 core configd.py: [ed8233ae-3d75-42e2-92a1-d7551c820c1f] request proxy status
Jan  1 19:38:57 core configd.py: [00835d1f-8609-402f-9571-dbf452ba7c6a] stopping proxy
Jan  1 19:39:29 core configd.py: [a010e876-002d-4f9e-be55-536e3974e2ad] request proxy status
Jan  1 19:39:30 core configd.py: [bf27749b-be67-428d-8b29-c2cc1b5ed6db] request proxy status
Jan  1 19:40:47 core configd.py: [61b46394-95f4-4166-b2f2-796568d90bf7] retrieve firmware execution status
Jan  1 19:40:47 core configd.py: [ba81f02f-db1b-45c3-8760-0a96c9241196] view remote packages
Jan  1 19:40:49 core configd.py: [1f4ab9af-58ce-45bb-8934-d420b9ab3895] view local packages
Jan  1 19:40:49 core configd.py: [b8ec1f37-14e3-4d13-b2b1-701ede2668ab] Retrieving changelog index
Jan  1 19:41:42 core configd.py: [bd6c16b3-c6c2-4ebd-a31f-7150c4e9039d] reinstalling firmware package squid3
Jan  1 19:41:42 core configd.py: message bd6c16b3-c6c2-4ebd-a31f-7150c4e9039d [firmware.reinstall] returned OK 
Jan  1 19:41:43 core configd.py: [79ab9a5c-3db0-4eca-bd16-2e3dfe83f891] retrieve upgrade progress status
Jan  1 19:41:43 core configd.py: [4d38ede7-f960-4c9e-b3c3-36cdd1db8c0a] retrieve upgrade progress status
Jan  1 19:41:44 core configd.py: [76af7acb-d7f8-4741-8f37-a2c37a91f2bf] retrieve upgrade progress status
Jan  1 19:41:44 core configd.py: [a498be0a-11b3-4d6f-a5c6-116c4f08dc45] retrieve upgrade progress status
Jan  1 19:41:45 core configd.py: [4a3054d0-c23a-45f2-b7f7-f6afee9b2217] retrieve upgrade progress status
Jan  1 19:41:46 core configd.py: [ab858c84-f9f5-4e39-84d8-48f911102806] retrieve upgrade progress status
Jan  1 19:41:46 core configd.py: [27154d2f-39cf-4719-a28f-0d054413e623] retrieve upgrade progress status
Jan  1 19:41:47 core configd.py: [156131e2-2004-4c13-833e-d1ae44634873] retrieve upgrade progress status
Jan  1 19:41:48 core configd.py: [a60e324a-f285-424c-9fed-429cab1d68f6] retrieve upgrade progress status
Jan  1 19:41:48 core configd.py: [8c4b28fd-5819-4a00-a3b8-b770831d4b5e] retrieve upgrade progress status
Jan  1 19:41:49 core configd.py: [e20f24cf-96d3-4e79-9fc9-938243ee189f] retrieve upgrade progress status
Jan  1 19:41:50 core configd.py: [0d0a7d1f-1e42-42a7-9c50-bd1e3b2540b4] retrieve upgrade progress status
Jan  1 19:41:50 core configd.py: [a1cc8228-530f-4d4f-9156-35605b4c7cb1] retrieve upgrade progress status
Jan  1 19:41:51 core configd.py: [7f535af2-9acc-4d09-942b-064a1b48ed2d] retrieve upgrade progress status
Jan  1 19:41:51 core configd.py: [4a1fd499-60cd-4887-ae8c-1a6d1adbcc0e] retrieve upgrade progress status
Jan  1 19:41:53 core configd.py: [4c61f8f6-79eb-4ab6-bbf9-c482a45e8d53] retrieve upgrade progress status
Jan  1 19:41:53 core configd.py: [951e9440-f2db-43c6-b0c3-a6d754ffc8c7] retrieve upgrade progress status
Jan  1 19:41:54 core configd.py: [607926d1-13ad-4ec3-b85b-9a84f39e7c07] retrieve upgrade progress status
Jan  1 19:41:55 core configd.py: [be193a78-e33f-4dcd-a863-7d3a36e02af2] retrieve upgrade progress status
Jan  1 19:41:55 core configd.py: [06943e39-a985-4206-84b7-04feffc7b409] retrieve upgrade progress status
Jan  1 19:41:56 core configd.py: [1c5bde05-0a5a-401f-abc7-8b14afb787ad] retrieve upgrade progress status
Jan  1 19:41:57 core configd.py: [48850a0a-6a36-4481-9d86-6a26007908b3] retrieve upgrade progress status
Jan  1 19:41:57 core configd.py: [817515ed-1b0e-40f7-8e64-4be6c5435b8c] retrieve upgrade progress status
Jan  1 19:41:58 core configd.py: [20fc3a56-6d82-47e3-bd71-e89884fa7349] retrieve upgrade progress status
Jan  1 19:41:58 core configd.py: [381c7808-c3ea-4731-b80a-eeda01709532] retrieve upgrade progress status
Jan  1 19:41:59 core configd.py: [f46ccdb8-628b-4cc6-b37f-98e668fdc98f] retrieve upgrade progress status
Jan  1 19:42:00 core configd.py: [0b6a8351-ed4d-4fbb-a31a-850d3b195c38] retrieve upgrade progress status
Jan  1 19:42:00 core configd.py: [9bc80350-c9a9-43d5-987e-afedde022eb4] retrieve upgrade progress status
Jan  1 19:42:01 core configd.py: [e269d699-94c1-4ff0-8ae9-aceca1c9dd7e] retrieve upgrade progress status
Jan  1 19:42:02 core configd.py: [af0051b7-ef61-47a4-856f-95b50b29399c] retrieve upgrade progress status
Jan  1 19:42:02 core configd.py: [ff14f284-fb5e-47d2-af51-f5fbc7b6ddcc] retrieve upgrade progress status
Jan  1 19:42:03 core configd.py: [5f370f89-fc76-4664-8061-28bdfacff544] retrieve upgrade progress status
Jan  1 19:42:04 core configd.py: [0c75c659-b41b-4a36-851a-568f124adf63] retrieve upgrade progress status
Jan  1 19:42:04 core configd.py: [e367157e-0d74-4e2e-838d-a38048d066c7] retrieve upgrade progress status
Jan  1 19:42:05 core configd.py: [5e13fc8b-a564-459e-aba6-7e38e72acd95] retrieve upgrade progress status
Jan  1 19:42:06 core configd.py: [fc05e9b0-7269-4f51-835a-2858cc007a04] retrieve upgrade progress status
Jan  1 19:42:06 core configd.py: [b7c71920-d95d-41cd-acae-d9e569468702] retrieve upgrade progress status
Jan  1 19:42:07 core configd.py: [3fae1eec-3086-46eb-8d55-c8e528292973] retrieve upgrade progress status
Jan  1 19:42:07 core configd.py: [f3cfa7b2-57f9-4e79-8b34-4d7c58337d42] retrieve upgrade progress status
Jan  1 19:42:08 core configd.py: [4dbae1b3-a23a-4233-bb98-701f4dc91261] retrieve upgrade progress status
Jan  1 19:42:09 core configd.py: [8fc3358e-ff1a-475a-a390-26ff699876e6] retrieve upgrade progress status
Jan  1 19:42:09 core configd.py: [a6a3ff33-a5f1-43f6-8f68-aa6fc2db3308] retrieve upgrade progress status
Jan  1 19:42:10 core configd.py: [a6a3c053-6b68-4590-a99b-4c78753b9012] retrieve upgrade progress status
Jan  1 19:42:11 core configd.py: [5394558f-2041-4d80-a663-d77ab2161327] retrieve upgrade progress status
Jan  1 19:42:11 core configd.py: [d2268a76-f724-40fe-8b69-7a512bc944c0] retrieve upgrade progress status
Jan  1 19:42:12 core configd.py: [467b4df3-d6c4-4f70-a6cf-21de13034f8b] retrieve upgrade progress status
Jan  1 19:42:13 core configd.py: [bc977957-8487-4a1a-93e3-c76a817f60be] retrieve upgrade progress status
Jan  1 19:42:14 core configd.py: [49eba421-3f18-4746-b15e-6dabb71b3535] retrieve upgrade progress status
Jan  1 19:42:15 core configd.py: [3369be38-8629-4faf-ac94-980156b8da72] retrieve upgrade progress status
Jan  1 19:42:16 core configd.py: [be46e067-417c-4527-a76b-d41bc2f1216c] retrieve upgrade progress status
Jan  1 19:42:17 core configd.py: [9ea010c8-39f0-4558-a300-eaf3781547da] retrieve upgrade progress status
Jan  1 19:42:17 core configd.py: [123cff90-977f-4033-a3a1-a45f89ce329f] retrieve upgrade progress status
Jan  1 19:42:18 core configd.py: [c6fc76db-e511-4982-b2a0-025d6ec51081] retrieve upgrade progress status
Jan  1 19:42:19 core configd.py: [3c986a46-1e8b-4336-82d0-c57a4c0c35aa] retrieve upgrade progress status
Jan  1 19:42:20 core configd.py: [7f21e6a4-89c1-4474-9593-da73978b5d2b] retrieve upgrade progress status
Jan  1 19:42:20 core configd.py: [e5a43978-a12f-4816-a521-6ac36478585b] retrieve upgrade progress status
Jan  1 19:42:21 core configd.py: [df93fef7-99ad-4455-b3db-0a379609c7ed] retrieve upgrade progress status
Jan  1 19:42:22 core configd.py: [643df2e1-df43-463e-8f8d-1e181eaa1cf1] retrieve upgrade progress status
Jan  1 19:42:23 core configd.py: [020ff207-b1a3-4179-825a-715254111b62] retrieve upgrade progress status
Jan  1 19:42:23 core configd.py: [d35946e7-fc75-4aea-8452-c2993f208667] retrieve upgrade progress status
Jan  1 19:42:24 core configd.py: [6c88f8c0-1cb8-412e-8097-e980e79b1d18] retrieve upgrade progress status
Jan  1 19:42:24 core configd.py: [f113fc0f-e29d-4673-add9-a9b289c3c084] view remote packages
Jan  1 19:42:25 core configd.py: [cfb455e6-8d1c-499c-8464-33bffee728d1] view local packages
Jan  1 19:42:25 core configd.py: [6a83807c-d1cb-465e-9496-fe654249a521] Retrieving changelog index
Jan  1 19:45:17 core configd.py: [d68906eb-dd79-43af-b28a-42d3c8baa2cb] request proxy status
Jan  1 19:45:19 core configd.py: [cec80068-e726-4dae-9b13-40271996295f] starting proxy
Jan  1 19:45:21 core configd.py: [446bf478-bbd6-499f-ae37-4d8e4d4e1546] request proxy status
Jan  1 19:45:21 core configd.py: [ebd2d7ff-bda1-49f7-815d-65bd5b7ef048] request proxy status
Jan  1 19:54:36 core configd.py: [d456e8eb-5964-4ba9-87f4-6795a3f5d42d] request pfctl byte/packet counters
Jan  1 19:55:43 core configd.py: [4c097108-26da-46f0-bad2-ad17c989a634] New IPv4 on pppoe0
Jan  1 19:55:44 core configd.py: [645e4aea-031d-4c23-9c4c-cc38ae77fc33] generate template OPNsense/Filter
Jan  1 19:55:44 core configd.py: generate template container OPNsense/Filter
Jan  1 19:55:45 core configd.py:  OPNsense/Filter generated //usr/local/etc/filter_tables.conf
Jan  1 19:55:45 core configd.py: [ad959a05-7574-405c-86c7-b606fb5441cd] refresh url table aliases
Jan  1 19:55:45 core configd.py: [60bffc25-c74e-4d44-86b9-52e23808a1e2] generate template *
Jan  1 19:55:45 core configd.py: generate template container OPNsense/Auth
Jan  1 19:55:45 core configd.py: generate template container OPNsense/Captiveportal
Jan  1 19:55:45 core configd.py: message ad959a05-7574-405c-86c7-b606fb5441cd [filter.refresh_aliases] returned OK 
Jan  1 19:55:45 core configd.py: generate template container OPNsense/Cron
Jan  1 19:55:45 core configd.py: generate template container OPNsense/Filter
Jan  1 19:55:45 core configd.py: generate template container OPNsense/IDS
Jan  1 19:55:46 core configd.py: generate template container OPNsense/IPFW
Jan  1 19:55:47 core configd.py: generate template container OPNsense/Macros
Jan  1 19:55:47 core configd.py: generate template container OPNsense/Monit
Jan  1 19:55:47 core configd.py: generate template container OPNsense/Netflow
Jan  1 19:55:47 core configd.py: generate template container OPNsense/Sample
Jan  1 19:55:47 core configd.py: generate template container OPNsense/Sample/sub1
Jan  1 19:55:47 core configd.py: generate template container OPNsense/Sample/sub2
Jan  1 19:55:47 core configd.py: generate template container OPNsense/Syslog
Jan  1 19:55:48 core configd.py: generate template container OPNsense/WebGui
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/sudoers.d/opnsense
Jan  1 19:55:48 core configd.py:  * generated //etc/pam.d/sshd
Jan  1 19:55:48 core configd.py:  * generated //etc/motd
Jan  1 19:55:48 core configd.py:  * generated //etc/pam.d/system
Jan  1 19:55:48 core configd.py:  * generated //var/etc/lighttpd-api-dispatcher.conf
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/captiveportal.conf
Jan  1 19:55:48 core configd.py:  * generated //etc/rc.conf.d/captiveportal
Jan  1 19:55:48 core configd.py:  * generated //var/cron/tabs/nobody
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/filter_tables.conf
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/suricata/suricata.yaml
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/suricata/rules/OPNsense.rules
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/suricata/rules.config
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/suricata/classification.config
Jan  1 19:55:48 core configd.py:  * generated //etc/rc.conf.d/suricata
Jan  1 19:55:48 core configd.py:  * generated //etc/newsyslog.conf.d/suricata
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/suricata/rule-updater.config
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/suricata/reference.config
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/suricata/custom.yaml
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/ipfw.rules
Jan  1 19:55:48 core configd.py:  * generated //etc/rc.conf.d/ipfw
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/monitrc
Jan  1 19:55:48 core configd.py:  * generated //etc/rc.conf.d/monit
Jan  1 19:55:48 core configd.py:  * generated //etc/rc.conf.d/flowd
Jan  1 19:55:48 core configd.py:  * generated //etc/rc.conf.d/flowd_aggregate
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/netflow.conf
Jan  1 19:55:48 core configd.py:  * generated //etc/rc.conf.d/netflow
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/flowd.conf
Jan  1 19:55:48 core configd.py:  * generated //tmp/template_sample/simple_page.txt
Jan  1 19:55:48 core configd.py:  * generated //tmp/template_sample/sub1/example_sub1.txt
Jan  1 19:55:48 core configd.py:  * generated //tmp/template_sample/sub2/example_sub2.txt
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/syslog-ng.conf
Jan  1 19:55:48 core configd.py:  * generated //etc/newsyslog.conf
Jan  1 19:55:48 core configd.py:  * generated //usr/local/lib/php.ini
Jan  1 19:55:48 core configd.py:  * generated //usr/local/etc/php.ini
Jan  1 19:55:51 core configd.py: [121e7b49-10ab-4105-b7ad-d9cff5d92cfd] restarting cron
Jan  1 19:57:44 core configd.py: [09a1dcbb-2dc3-4746-a73f-49b2a4006a9d] request pfctl byte/packet counters
Jan  1 19:57:50 core configd.py: [0ed1bcd6-ea92-4f3d-83a1-898c06c834ca] request pfctl byte/packet counters
Jan  1 19:58:15 core configd.py: [1c3e32a7-0e2b-4110-8811-9efd68666c8d] Reloading filter
Jan  1 19:58:16 core configd.py: [fb09bff5-c55d-46f0-af88-be31cf9e0a46] generate template OPNsense/Filter
Jan  1 19:58:16 core configd.py: generate template container OPNsense/Filter
Jan  1 19:58:16 core configd.py:  OPNsense/Filter generated //usr/local/etc/filter_tables.conf
Jan  1 19:58:16 core configd.py: [f04c557c-59d9-433b-9d2d-39d6a69a74cf] refresh url table aliases
Jan  1 19:58:17 core configd.py: message f04c557c-59d9-433b-9d2d-39d6a69a74cf [filter.refresh_aliases] returned OK 
Jan  1 20:03:58 core configd.py: [05a92174-82fb-4d2d-8227-60339d60f04f] retrieve firmware execution status
Jan  1 20:03:58 core configd.py: [a2d487f7-ab2b-468a-ab3f-e1a11d2d4e08] view remote packages
Jan  1 20:04:02 core configd.py: [afba165c-bac3-458f-a93d-09eb871efe70] view local packages
Jan  1 20:04:02 core configd.py: [f66bef20-f970-4c4f-ae28-6db7be4a9695] Retrieving changelog index
Jan  1 20:22:26 core configd.py: [68bdc9ef-5170-4b09-9a3c-c6eb06a0a4f7] request pfctl byte/packet counters
Jan  1 20:22:32 core configd.py: [38733bc6-3597-4a66-b36d-4a3ce6bd5df5] request pfctl byte/packet counters
Jan  1 20:22:38 core configd.py: [21e5ec44-fe07-4308-829a-1eff53c6d7e8] request pfctl byte/packet counters
Jan  1 20:22:44 core configd.py: [5fff5183-f677-4b9a-915c-a130b99e07ca] request pfctl byte/packet counters
Jan  1 20:22:50 core configd.py: [b36426c4-1df6-4a80-8bda-f96c67ca15c2] request pfctl byte/packet counters


[guest19757]

Hello,

To answer you first question:

My understanding was that configuration was parsed from /conf/config.xml, is that not the case?

Your settings are stored in /conf/config.xml, then configd parses it and generates a conf file for squid to utilize here /usr/local/etc/squid/squid.conf. You should see that in Configd.log, **** generated /usr/local/etc/squid/squid.conf.

I have tried removing all settings from /conf/config.xml between the <squid> flags, and deleting all files in /usr/local/etc/squid and rebooting. After which I have done a really basic setup with the local network, HTTP proxy only, and receive the same error message.

This should have really have been done through the WebGUI, to avoid accidentally borking /conf/config.xml. Bare minimum, backing up /conf/config.xml before making any manual changes. Now it's harder to distinguish if it's a 'User error' or a bug in Opnsense.

Nevertheless, if removing and installing Squid3, in fact removes all the changes you've made. Simply ticking it on, not making any more changes than the basic defaults. Does it work?

P.S. Looking at your conf

[guest19757]

hello,

Is that complete squid.conf? Seems like it's missing some bits there?

Here's a sample conffile, don't copy and paste:

Code: [Select]

#
# Automatic generated configuration for Squid.
# Do not edit this file manually.
#



# Setup regular listeners configuration
http_port 10.0.0.1:3128 


acl ftp proto FTP
http_access allow ftp


# Setup ftp proxy

# Rules allowing access from your local networks.
# Generated list of (internal) IP networks from where browsing
# should be allowed. (Allow interface subnets).
acl localnet src 10.0.0.0/24 # Possible internal network (interfaces v4)
# Default allow for local-link and private networks
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

# ACL lists

# ACL - Blacklist - User defined (blackList)
acl blackList url_regex *

# ACL - Remote fetched Blacklist (remoteblacklist)

# ACL - Block browser/user-agent - User defined (browser)

# ACL - SSL ports, default are configured in config.xml
# Configured SSL ports (if defaults are not listed, then they have been removed from the configuration!):
acl SSL_ports port 443 # https

# Default Safe ports are now defined in config.xml
# Configured Safe ports (if defaults are not listed, then they have been removed from the configuration!):
# ACL - Safe_ports
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 282 # video sites
acl CONNECT method CONNECT

# ICAP SETTINGS
# disable icap
icap_enable off

# Pre-auth plugins
include /usr/local/etc/squid/pre-auth/*.conf

# Authentication Settings





#
# ACL list (Deny) blacklist
http_access deny blackList




# Deny requests to certain unsafe ports

http_access deny !Safe_ports
# Deny CONNECT to other than secure SSL ports

http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
http_access deny to_localhost


# Auth plugins
include /usr/local/etc/squid/auth/*.conf

#
# Access Permission configuration:
#
# Deny request from unauthorized clients

#
# ACL - localnet - default these include ranges from selected interfaces (Allow local subnets)
http_access allow localnet

# ACL - localhost
http_access allow localhost

# Deny all other access to this proxy
http_access deny all
# Post-auth plugins
include /usr/local/etc/squid/post-auth/*.conf

# Caching settings
cache_mem 256 MB
cache_dir ufs /var/squid/cache 4072 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/squid/cache

#
# Add any of your own refresh_pattern entries above these.
#


refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

# Squid Options

access_log stdio:/var/log/squid/access.log squid
cache_store_log stdio:/var/log/squid/store.log
# Disable via Header
via off
# URI hanlding with Whitespaces (default=strip)
uri_whitespace strip
# X-Forwarded header handling (default=on)
forwarded_for on
# Disable squid logfile rotate to use system defaults
logfile_rotate 0
# Define visible email
cache_mgr admin@lhpmail.us
error_directory /usr/local/etc/squid/errors/en-us


[jordan_richardo]

Gotcha.

I edited a copy of the config.xml and restored the original after trying it and it not working.

Tried just ticking the proxy on, and same error. I have a feeling that you are onto something with the squid.conf file, it seemed bleak to me and I thought perhaps there is another file somewhere that is actually in use. I'm thinking it might have been damaged along the way.

I WINSCP'd that copy and uploaded it directly to get the exact copy.

Is there a possibility that I could take a copy of squid.conf from a fresh install and copy it over? The router is in production and I'm not terribly exited about trying to start from scratch lol. If it would just be a matter of squid not starting, that is a fine risk for me. The router not coming back up however would be rough.

Thanks for your help with this
Jordan

[guest19757]

Is there a possibility that I could take a copy of squid.conf from a fresh install and copy it over? The router is in production and I'm not terribly exited about trying to start from scratch lol. If it would just be a matter of squid not starting, that is a fine risk for me. The router not coming back up however would be rough.

You could do that but that wouldn't solve the root cause, you might want to do a audit check! System -> Firmware -> Updates: Audit Now.

[jordan_richardo]

AHA!

I did the "Audit Health" option, and received a list of missing files all centered around the proxy templates:

Code: [Select]

***GOT REQUEST TO AUDIT HEALTH***
Check for and install missing package dependencies
Checking all packages: .......... done
Detect installed package files with invalid checksums
Checking all packages: ....
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/+TARGETS
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/auth.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/ca.pem.id
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/cache.active
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/externalACLs.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/newsyslog.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/nobumpsites.acl
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/post-auth.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/pre-auth.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/rc.conf.d
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/snmp.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/squid.acl.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/squid.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/squid.user.local_auth.conf
opnsense-18.7.9: missing file /usr/local/opnsense/service/templates/OPNsense/Proxy/wpad.dat
Checking all packages......... done
***DONE***
...So I guess the next question is what do I do with this information now? haha Is the proxy still a "Package" whereby I can uninstall/install like the good ol' PFsense days or is it completely integrated at this point? I suppose I could copy these files from a fresh install as well...

Thanks!
Jordan

[guest19757]

I'm not sure of a recommended way of solving this short of reinstalling which you are not enthusiastic of doing. I've only been using Opnsense for roughly a week and forget how long I've been with freebsd. I wonder if simply running updates again would fix that?

[guest19757]

Btw, it shouldn't take no time? During install, just import you config? Setup you nic cards, done?

That is, if, running update fails.

[fabian]

you may reinstall the opnsense package (pkg install -f opnsense), then the missing files should be there - looks a bit like your installation is broken because it is not very common that system files are missing. Is your file system ok or is it broken (for example a missing inode which contains the directory elements) which may happen due to a power outage.

[jordan_richardo]

Hey Guys,

Thanks for the great info. I ran the "pkg install -f opnsense" and all of the missing files have shown up . I will test tonight and let you guys know.

Thank you again for all the help, much appreciated!
Jordan

[franco]

Reinstalling a package also works from the GUI: System: Firmware: Packages... look for "opnsense" ... click "reinstall".


Cheers,
Franco

[jordan_richardo]

Gentlemen, we are back in action .

I do still seem to get the error if I enter a website (for example yahoo.com) and it then redirects to the HTTPS version. Manually typing in the URL with HTTPS does work though. Combing through to see if I messed some obscure setting up in my messing around.

Thank you all again!
Jordan