Recovering an Edge Gateway


#1

Hello, we have a customer that got one of your boxes and we tried upgrading it from the state it was received in to 4.6 using the install guide. The installer appeared to disable one of the ports and we no longer have SSH access. We can’t connect to the platform using a Niagara Workbench either, though it seems like Niagara might be running? We just get Connection Refused messages.

Is there anyone that could help us with this?


#2

Run the command

sudo docker ps -a

This will give you a list of all the versions of Niagara possibly running. You might not have stopped the 4.2. installation. You will be able to see this by the running state and the image name containing 4.2, If 4.2 is still running, you will need to stop it and then make sure your new 4.6 install is in the running state.

The installer has no effect on IP addresses or SSH connections. If you were configuring your ports via DHCP and a reboot occurred you could have gotten a new IP address assigned. You may have to determine the new IP address with a keyboard and monitor or through any IT staff that may be monitoring the gateway on the network.


#3

So we plugged it into a network and watched the DHCP clients table from the router and nothing seemed to come up. None of the previously used IP addresses work either. Any other strategies you can think of to get back into the thing?


#4

If you can’t determine the IP address from any networking tools you have available, I would suggest a keyboard and monitor to login and retrieve the current IP configuration.


#5

Trying that next. For the Dell Edge Gateway 3000 model, do you know if a USB -> HDMI display adapter would work? It doesn’t have an HDMI port like the 5000. Can’t seem to find any documentation on the product on Dell’s site, the support link errors out.


#6

Only the 3003 has an HDMI port. Assuming it wasn’t removed at installation, there is generally a sticker with the mac addresses which could be used to match with an IP scanner if DHCP is present.

I am unfamiliar with any other recovery process for the 3000 series without a monitor support other than a re-installation of the operating system. Those instructions if available would likely need to come from Dell if they exist.

Here is some additional info on the default IP addresses for the gateway series.


#7

No luck on port scanners, can’t find anything at all. Nothing pops in when connected to a DHCP server.

If I were to recover it using an Ubuntu 16 Core recovery image, do you think I’d be good to go to install Niagara? Or is there other software preloaded on this thing that may be necessary for that Niagara installation to run smoothly?


#8

If you restore from the core image, you will need update all the snaps before attempting to install the Niagara. I would also recommend during the installation process to use the options to install docker from the internet.

Update all snaps with the command:

sudo snap refresh

I would recommend rebooting after this is done to make sure they are are running the latest update that just got installed.


#9

Installer question: should I assume there are no updates for the Niagara image available on the internet? Seems to ask a bunch. I’ve been denying them and just saying to install what’s included (except Docker).


#10

As an additional question: successfully connected to the platform, then tested a reboot (sudo init 6), and waited a bit. Was able to serial back in to the system with the same IP configuration, but the Niagara service seemingly did not start on its own. Is there an additional step I need to take to get that to start automatically?

I guess a recommended* additional step is more accurate. I could probably fenagle some Linux stuff to get that script to auto-start, but was wondering if there was a typical, expected procedure to follow for this.


#11

Yes, the niagara images are available online. There are a few options in the installer that allow you to download these versions without needing to update the installer if a later version is released that you need to update to. These are the selection version option and the install latest. Latest is considered the development version and not officially supported so is not intended to be used unless directly asked to download this version for troubleshooting purposes.

Towards the end of the installer, it asks if you wish to allow docker to manage the auto starting of Niagara. If you do not say yes, then you will need to do some sort of linux service manual setup which may be problematic with Ubuntu Core’s security model so it is not recommended. If for some reason docker did not start on reboot, then Niagara won’t stat since it is tied to that service.


#12

Gyah! Here’s my installation output haha:

Would you like Docker to manage the Niagara’s start/restart capability(Y,N)?In order for Niagara to correctly access TCP/IP configuration data correctly, the network stack must be configured with the NetworkManager snap.
Checking the status of NetworkManager…

So I dunno it looks like I missed it somehow. There was definitely a point at which I stuttered and hit enter twice. Might have been there.

Any ideas on how I might get that back? Other than blow away the whole thing again and start over? I assume running the install script again on top of the existing installation isn’t the greatest idea.


#13

Nothing bad or wrong about re-installing. You can select option 4 to re-install the same version and then make sure to select the option to preserve all user data.


#14

This does create a backup of the original installation so it is a good idea to go back and clean up all unnecessary backups.


#15

Happen to know where those backups are stored? This is a fresh installation so I doubt it’s a significant file size, but, curious as to where it is haha.


#16

To get a list type:

sudo docker ps -a

The backups will be denoted by the name “backup” as well as a time stamp for when they were made.

To manually clean them up, you can run the command:

sudo docker rm -v

The -v flag will remove all the persistent data that was used to pull existing user profiles and station data into the new installation. If you don’t have that flag it’ll only remove the niagara software instance but leave all that data behind for future reference and/or restoring. It is quite a process to access after the container reference is removed so it’s best to remove them at the same time when you are ready to do that.


#17

So, from my latest log:

Would you like Docker to manage the Niagara’s start/restart capability(Y,N)?Y
Unless explicitly stopped or docker itself is stopped, Docker will attempt to always restart the Niagara service.

Power cycled it and the Niagara service never came back again :confused:

Anywhere else I can check to verify Docker’s setup properly for this?


#18

Bleh yeah, I think the Niagara container has the correct restart policy, but the Docker daemon is not starting on reboot.


#19

There has been occasions with network switching where the first power cycle docker seems to fail to establish its network stack.

This restarts docker:
sudo snap disable docker
sudo snap enable docker

You may also want to run and update on all current snaps to make sure everything is up to date which sometimes helps stability with the startup process. This does require internet access.

sudo snap refresh


#20

Welp, that was it. Rebooted again and it’s been working fine haha.

Thanks for all the help with this one. Pretty neat product.