So since you chose to ignore it you I assume you don’t think releasing a game for a market 30x the size of the one it’s designed for could lead to any issues?
Which of those regions are run by Amazon, who owns majority of the whole internet ? Brain on pls.
Where can I buy some shares in this internet you speak of?
If you’re an IT BA for 15 years, you should have the solution that their own technicians haven’t thought about. Contact them immediately and save us all.
I do, but that’s called planning, and as you chose to ignore the part where I said “its not about the mess up, its about how they handle it”.
If they messed up, and then said:
hey guys, we messed up. If you want a refund please contact us here: … Were implementing a new region and we have taken your feedback on board. We will be cloning the entire region so no-one loses their purchases and progress. They won’t stay in sync, but please choose which region to play on going forward. Unused accounts on the other region will be disabled after 1 month
See look i just solved the entire forums problem. Wasn’t so hard was it.
You didn’t wolve anything though, it’s easy to just make up solutions without having to deal with pesky logistics lmao.
Hold on, lemme solve world hunger quick:
hey guys, here’s some food for everyone that’s hungry and also you all have internet and clean water, sorry about you dying and stuff
damn that was easy without actually having to know anything about what it would take to implement that stuff!
You didn’t wolve anything though, it’s easy to just make up solutions without having to deal with pesky logistics lmao.
I mean you are suggesting that amazon dont have the ability to clone an environment? Because that would suggest they are unable to back and restore too? Which would be quite a big problem. It would be a massive worry for me if amazon are unable to do this, rather than unwilling. a really huge worry.
My comment is based on my experience that every single server environment has had for the last 10-20 years. Im not saying its easy, im saying its possible. just amazon dont care enough to do it, they already have our money and everything else is irrelevant.
For 8 years I worked for one of the largest airlines on the planet as a Regional LAN admin. Once we were copying data from our backup tapes to a different medium and it was a massive problem and even with our full breadth of engineering staff and HP themselves sending in techs it was still a nightmare.
The idea that they could just “copy over” an “environment” is just laughably simplistic.
Cloning seems like an obvious and easy solution, but:
- it’ll take a long(-er) maintenance period and people are already rioting because of the 3h frames in the morning (okay this would only be a small shitstorm)
- after the cloning process, you’d have to decide, if you want to use region A or B and disable/delete the other one → what if you can’t coordinate this with all of your friends/guildmates and end up in different regions (again shitstorm)
- many accounts are linked to digital purchases done through Steam, that can’t easily by duplicated I guess. Just imagine it would look like a key you want to redeem and Steams says “already in use”. That’s exactly what would happen, by cloning the servers. I suspect that’s also one of the facts why they can’t easily clone all the founders packs. Maybe if one account is deleted and the other one is then persistent (by choosing region A or B) this could be done, but everything would’ need to be in sync with Steam to avoid duplication…this might even have fiscal implications and so on
- because of all the “sync issues”, I believe it is not possible to run just clone and run both regions at the same time with exactly the same accounts. You’d have to implement a mechanism to let the players decide upfront, which region the want to use, then clean up the region servers, then start the servers (and fingers crossed, everything goes well)
- …I guess there is a whole lot more to consider
In general your solutions sounds great, but if you’re think about it, it sadly gets more and more difficult.
Your right, the IT systems of an airline clearly match that of amazon, or for that matter, any software house worth their money. Obviously, an airline put a lot of resources into their IT… i mean just take a look at their websites… its really marvellous stuff.
I’m done with this conversation, i dont care for your opinion on the matter, your experience clearly shows your know what you are talking about. Amazon have handled this poorly, especially in the aftermath. I dont know what else there is to discuss.
Agree, but I mean, its already terrible, its only going to get worse.
True, but at least people have the option and dont lose anything. The first week might take a little time for people to settle. After that it should iron itself out.
Yes, but i doubt it would re-redeem from steam, it should already be available in inventories. If it does, that would be strange but a valid concern. Should be possible to iron out from AGS though.
Trading of items on the region not used. EUW person trades EUC person, and then they do visa versa on the other region. Also potential for AH problems with people selling cheap. However, these can also be worked out and resolved.
I’m not saying it free from problems but the alternative is everyone loses everything, including what they paid for.
Do you really look at an airlines website and take this as an example how the whole IT must be structured.
Oh well, that’s really very simplistic (and I don’t believe you really look at it this way).
I give you a very simple example:
- lets assume you have a home network with just one PC and assigned a fix IP to this machine
- now you clone the PC, because some family member also wants to play LA with you, but doesn’t want to install Windows, Steam, the game…you get it
- now you boot up this “Clone PC” and…you got a conflict in your network, because two machines got the same IP
Well yes this is a really over simplified example. But now scale that up by…let’s say a million (which might come close to an MMO serverfarm) and you are getting close to the problems AWS might face when just simply “cloning the whole EU region”, or also taking the example of Thenatural, cloning the backend infrastructure of an airline.
In the end it is just a matter of time.
Just because it is so complicated, let’s assume Amazon says: “we found a solution, but it’ll take two month”, or just for arguments sake, we assume they are really, really quick and get it done in one month (which I think impossible, but we’re just fantasizing, right?)
How do you think the reaction will be?
They already announced, that they are working on the best possible quick(!) solution and even after 48h people are rioting, because they are not quicker.
Already they have no ETA for setting up a whole new region. Do you think they could give us an ETA of cloning and syncing everything quicker?
You would actually be surprised. It’s one of the most demanding jobs around and talented engineers do really well for themselves.
Cloning servers happens on millions of servers across the internet every single day. I literally have one being cloned right now so I can test something against it. Yes cloning isnt a 100% clone, certain things change, but its a known way of working. If amazon cant do this then im shocked, its technology thats been around for several years.
As for time, presuming amazon actually hired talented people, It shouldn’t take that long for collective minds to work out solutions to the problems. Actual maintenance times would only be due to the size of the data… problem wise, they should have plans in place before they start. Going in blind would obviously be very bad. Planning it probably takes more time that executing it, except maybe because of the data size.
I mean your suggesting amazon dont have the ability to clone a server, something thats been available for several years. So I can only assume, in my sample size of 1 (you), that airlines dont really have their stuff in order.
“just clone the server”
oof
We are not talking about 1 server here 
I can also clone my own servers in no time hehe.
But just as an example, you might want to read this article:
Or just look at some of the diagrams showing the architecture. This is not Lost Ark, but you’ll get an idea of the scale of it. And this is without Steam involved.
If it was very simple, you need:
- 1 login server
- 1 “trade” server
- 1 gaming server
- 1 database server
- some other infrastructure stuff like load balancers and so on
An as before: now scale this up several times to get on a scale of cloning a whole region.
I’m not saying it is impossible, quite the opposite. But it takes a whole lot of time and testing. Because you need to clone everything, get it up and running, without anyone being able to connect of course. Take necessary steps for the re-configuration for the new region, test, test, test and then you’ll have it read.
I think Thenatural could make a far better assumption of the time necessary to do all this, but I’d say just out of my head, that even 2 month would be a very close window to do all that.
We also have enough money and people to solve world hunger issues, but somehow there are still hungry people on this planet.
You still need the competent people to do it, even if you have unlimited money 
Also, having proper launch with 20 people is different from servers getting hit by millions KEKW
well firstly, this is specific to a region. we already know that. And yes there will be multiple servers and services. Which is no different from bring up the new region, it will be lots of servers and services (presuming this isn’t cloud based, and i seem to remember they said it wasnt). Its the data layer for a single region, so really it isn’t that farfetched to be able to clone. Just google around:
AWS clone server/database/ec2
azure clone server/database/vm
ovh clone server/database/vm
They can all do it. I picked 2 cloud providers and one dedi provider. They all offer cloning. yet amazon are unable? come on.
Presuming it is not “cloud based”?
What do you think is the main field of Amazon? Cloud based infrastructure services.
Of course LA is running on the AWS machines.
I also can simply clone one of my AWS machines with some simple clicks. But I’m not running an MMO
I can’t find the post, but one of the CM’s confirmed that the game isn’t setup for AWS, so doesn’t use it. If it was using AWS this cloning wouldn’t even be a discussion tbh. It’s to do with the games architecture, which is fair enough
Edit: It could be using azure, but i’d be surprised if amazon were paying a competitor for something this size… maybe they are.