A week of error solving
Weekend is finally here. Got some time to blog more details about what's happening recently.
I knew this is going to be a busy week. But I didn't expect it to be this busy.
There was a major error in the system at work. Actually, I saw that error late last week. But I thought that its just a special case affecting one customer only. I didn't expect so many customers to be affected by the error until Monday afternoon when we suddenly get a surge in numbers of queries from customer.
So I spend the entire week trying to resolve the error. Monday was spend on finding out how big the impact was. Tuesday and Wednesday was spend trying to find a solution. And it was difficult for the 3 teams to agree on a solution. We have couple of workaround methods to solve the error. But every method would impact at least one team. And the other teams wouldn't agree on the solution if there is a big impact to their team's coding.
My team was easiest going one. Basically, whatever option we choose, my team would have quite some impact. Therefore, it doesn't really matter us much. But the other teams weren't so easy. It took several meetings and brainstorming to come up with the final agreed solution. Personally, I don't like this method of approch. Its not 'clean cut' enough. But since we can't get the rest to agree with other solution, I guess this would be the best solution.
Thursday and Friday was spend on identifying the affected customer and writing SQL query to extract the affected cases from the database. It wasn't easy writing the SQL. There are 1001 possible scenario. Almost impossible for a SQL query to identify all cases. I took more than a day to finally cover almost all scenario. I hope I didn't miss out any. The other team was waiting for my SQL query to continue their work. Feel quite bad making them wait so long for that query. But they also know that its a difficult query to write.
It was a busy week. Next week is no better.
Oh, need to go back office on Sunday evening to run some query and get the operators to run some jobs. I could use VPN at home, but I wasn't very comfortable with the idea. What if the VPN is down? What if the network is slow? I only got 1 1/2 hour timeframe. Better not take the risk. It would be safer to do it in office.
I knew this is going to be a busy week. But I didn't expect it to be this busy.
There was a major error in the system at work. Actually, I saw that error late last week. But I thought that its just a special case affecting one customer only. I didn't expect so many customers to be affected by the error until Monday afternoon when we suddenly get a surge in numbers of queries from customer.
So I spend the entire week trying to resolve the error. Monday was spend on finding out how big the impact was. Tuesday and Wednesday was spend trying to find a solution. And it was difficult for the 3 teams to agree on a solution. We have couple of workaround methods to solve the error. But every method would impact at least one team. And the other teams wouldn't agree on the solution if there is a big impact to their team's coding.
My team was easiest going one. Basically, whatever option we choose, my team would have quite some impact. Therefore, it doesn't really matter us much. But the other teams weren't so easy. It took several meetings and brainstorming to come up with the final agreed solution. Personally, I don't like this method of approch. Its not 'clean cut' enough. But since we can't get the rest to agree with other solution, I guess this would be the best solution.
Thursday and Friday was spend on identifying the affected customer and writing SQL query to extract the affected cases from the database. It wasn't easy writing the SQL. There are 1001 possible scenario. Almost impossible for a SQL query to identify all cases. I took more than a day to finally cover almost all scenario. I hope I didn't miss out any. The other team was waiting for my SQL query to continue their work. Feel quite bad making them wait so long for that query. But they also know that its a difficult query to write.
It was a busy week. Next week is no better.
Oh, need to go back office on Sunday evening to run some query and get the operators to run some jobs. I could use VPN at home, but I wasn't very comfortable with the idea. What if the VPN is down? What if the network is slow? I only got 1 1/2 hour timeframe. Better not take the risk. It would be safer to do it in office.
