ROBIN -  Open Source Mesh Network Forum Index ROBIN - Open Source Mesh Network
users community forum
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

My worst nightmare, what to do when nodes fail Nationwide

 
Post new topic   Reply to topic    ROBIN - Open Source Mesh Network Forum Index -> beta-1.5 Release Candidates
View previous topic :: View next topic  
Author Message
Ads






Posted: Sat Sep 23, 2017 11:45 am    Post subject: Ads

Back to top
hyperllc
User
User


Joined: 18 May 2008
Posts: 39
Location: St Augustine FL - Albany GA - Birmingham AL

PostPosted: Tue Feb 02, 2010 9:04 am    Post subject: My worst nightmare, what to do when nodes fail Nationwide Reply with quote

We have about 70 OM1P gateway nodes and 90 OM1P repeater nodes on one OM network throughout The States, Hawaii and Puerto Rico inside large touchscreen Kiosks.

Of the 65 nodes (2671) registered to the OM dashboard 41 (now 2683) are down today.
We still have about 95 MORE nodes to yet ADD to the dashboard that ARE plugged in right now Nationwide.

I have always had "Disable Automatic Upgrades" checked and "Use test firmware unchecked"

All 2683 are down today which I please need someone to explain to me how did these get upgraded in the first place.
It almost seems like they are upgrading when not attached to the dashboard yet.



1. Is this issue fixed so I can add the rest.

2. How do I fix the ones not showing up

3. How do you go back to 2671 on the nodes that may be at 2683 working or not.

4. Best way to re-flash or salvage. (Thousands were spent to on 4 guys Flights, rentals, hotels to deploy these over last 2 months. The OM1P's are locked up inside kiosk cabinets)

There are Mobile Broadband Routers providing internet to each an OM1P gateway as well as a Mini PC.
I have remote access via VPN to each Broadband Router and PC. Other PCs behind Repeater Nodes are currently unreachable.

What do we do from here.

Thank You
Chris


Last edited by hyperllc on Tue Feb 02, 2010 9:33 am; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
foxtroop11
Service Provider
Service Provider


Joined: 22 Mar 2009
Posts: 1168
Location: Ansbach, Germany and sometimes the States

PostPosted: Tue Feb 02, 2010 9:18 am    Post subject: Reply with quote

I would take a look here,

Only registered users can see links on this forum!
Register or Login on forum!



Then possibly call/email Mike at the information listed on the open-mesh.com website.

Something I've said before and will say again, and it's nothing against Open-Mesh and or the firmware as this should not happen. I honestly have no idea what happened in your case, but as an extra precaution if you have your network setup the way you want it with the firmware that you are comfortable with I highly suggest using a custom.sh file to disable /sbin/upgrade e.g chmod -x /sbin/upgrade.

This will remove any possiblity that your nodes would accidently upgrade without you first making the file executable again.
Back to top
View user's profile Send private message
hyperllc
User
User


Joined: 18 May 2008
Posts: 39
Location: St Augustine FL - Albany GA - Birmingham AL

PostPosted: Tue Feb 02, 2010 9:43 am    Post subject: Reply with quote

Thanks, I got that message emailed to me earlier and could not believe I was one of the <1%.

I can't figure out how this happened to our most critical network.

I am trying to figure what I did wrong to be effected.

The only thing I different on this network than others was not register nodes with the OM Dashboard right away. They have been up unregistered for a while. It seems as soon as I added some today - bang!

Mike says the fault was on their end.

Was this just coincidence as I added there was a bad file on server for just a bit or this was inevitable as soon a 2683 would try and upgrade again. Last week I lost a 2683 group strangely after adding to dash.
Back to top
View user's profile Send private message Visit poster's website
foxtroop11
Service Provider
Service Provider


Joined: 22 Mar 2009
Posts: 1168
Location: Ansbach, Germany and sometimes the States

PostPosted: Tue Feb 02, 2010 10:15 am    Post subject: Reply with quote

There was patch put in that elimanted the possiblity for the nodes to upgrade prior to updating. So I guess you had them on long enough without adding them to the dash which by default let them upgrade.

That stinks, but even what I recommened would not have worked since you put them all online without adding them to the dash. Pretty sure this has been taken care of now, but I think all 2671-? could have this happen if someone put them online without adding them to the dash.

I hope you get it sorted out.
Back to top
View user's profile Send private message
reverged
Skilled User
Skilled User


Joined: 29 Jul 2009
Posts: 169
Location: Los Angeles, CA

PostPosted: Wed Feb 03, 2010 4:28 am    Post subject: Reply with quote

hyperllc wrote:
Thanks, I got that message emailed to me earlier and could not believe I was one of the <1%.

I can't figure out how this happened to our most critical network.

I am trying to figure what I did wrong to be effected.

The only thing I different on this network than others was not register nodes with the OM Dashboard right away. They have been up unregistered for a while. It seems as soon as I added some today - bang!

Mike says the fault was on their end.

Was this just coincidence as I added there was a bad file on server for just a bit or this was inevitable as soon a 2683 would try and upgrade again. Last week I lost a 2683 group strangely after adding to dash.


If you are running a production network, get your own dashboard.
I learned the hard way. An apology does not pay for a truck roll.

It took very little time to get the orange mesh code up and running to the point where I could sever ties from open-mesh. I haven't looked back.

The open-mesh dashboard, unlike the nodes, is not reliable and not ready for prime time.

This has happened far too many times.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    ROBIN - Open Source Mesh Network Forum Index -> beta-1.5 Release Candidates All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
c d
e



Powered by phpBB © 2001, 2005 phpBB Group

Abuse - Report Abuse - TOS & Privacy.
Powered by forumup.it free forum, create your free forum! Created by Hyarbor & Qooqoa
Confirmed

Page generation time: 0.056