The round trip time (RTT) was short when the DB query and application server were in the same zone. When the DB zone was moved and it became different from the application server’s zone, the latency is increased from 1 ms to 6 ms. In terms of the time for a single query, there is no much difference between 1 ms and 6 ms. However, in case of hundreds of queries, problems arose.
Because the required time was short and the resources used for a single query were a few, it was not captured by the DB slow query log and resource usage. If we did not use the WhaTap solution, it was difficult to find the root cause.
- Dong-hoon Lee, developer of Data Service team of Opensurvey
- Dong-hoon Lee, developer of Data Service team of Opensurvey -
Opensurvey, a mobile research company, is collecting and processing 400,000 survey data or more for each month. Accordingly, the product development group has used WhaTap Monitoring for DevOps from 5 years ago. Especially, we are getting a lot of help in analyzing failures with the services.
Recently, the services are being used not only for simple monitoring but also for the recruitment process. With a single chart, we are evaluating the problem-solving abilities of applicants. Today, we met with Dong-hoon Lee, a developer of the Data Service team, and Hyun-min Park, a backend developer, who are from the Open Survey Product Development group and have been using the WhaTap solution in various fields for a long time. We listened to their stories.
- - Dong-hoon Lee (developer of Data Service team), Hyun-min Park (Feedback Backend developer)
Would you please introduce your company?
Opensurvey is a company that collects and analyzes the data from various customers through mobile research to provide insights for companies with business concerns.
We think the difference between our company and other research companies lies in “technology.” Opensurvey has 200,000 mobile panels that can quickly participate in surveys anytime, anywhere through the technology that can send surveys only to the target panels. We can randomly select representative panels proportional to gender, age, and regional population distribution. For example, we can search for panels with specific conditions, such as consumers who visited large marts within the last 3 months and spent more than 50,000 won (or 38 USD) to send a survey. We can exclude consumers who have recently received surveys on similar topics or surveys from competitors in the same industry, to send surveys to panels with no response bias.
If there is one axis to accumulate consumer data through a sophisticated survey targeting system, there is another axis to provide professional analysis tools or manage and operate the data so that the accumulated data can be utilized in many fields. Through a self-developed survey data analysis tool called “Open Analytics,” large-scale survey data analysis results are provided in real time on a web basis, and the credit card payment records or food diary data (diet data in Korea) of panels, are being accumulated to regularly distribute analysis data. If you consider us one of research firms, you may not express some concern about the story of Opensurvey's long-term use of the WhaTap solution as a DevOps monitoring tool. We are collecting and analyzing vast amounts of consumer data, so we could participate in this interview with WhaTap Labs today.
Would you please introduce you two and your department that is using the WhaTap solution?
Dong-hoon Lee, developer of Data Service team (hereinafter “Dong-hoon”): Hello. I am Dong-hoon Lee who is a development lead in the Open Survey Data Service team. Currently, we are using the WhaTap solution on the backend. Thanks to the rapid growth of our company, we believe that a separate operation team will be organized in the near future if the size of the company grows further.
Hyun-min Park, Feedback Backend developer (hereinafter “Hyun-min”): I am Hyun-min Park who is in charge of developing the backend for “Feedback,” which is a new service that our company will release at the end of this year. For reference, “Feedback” is a new SaaS-based product that allows each company to collect and analyze CX data for customers by partially using the collection and analysis technology for survey-based consumer data, which is the core technology of Opensurvey.
Hyun-min Park, Feedback Backend developer (left), Dong-hoon Lee, developer of Data Service team (right)
What made you choose the WhaTap solution?
The Product Development group of Opensurvey has a lot of excellent members. Opensurvey’s management is thinking a lot to improve the efficiency of human resources so that they can focus more on their duties. If possible, they want to do that with an aid of solution. As part of that, they adopted a monitoring tool to reduce the resources of the Operation team.
Before use of the monitoring tool, they reviewed various products including the WhaTap solution. In the end after thinking over various criteria, we chose the WhaTap solution. Company J’s products were not cloud-based. In case of company D’s solution, the micro service architecture (MSA) was drawn well, but we felt like it was about 2% short of the features we expected. When a failure occurs, we need to figure out what the problem is, but we don't have to investigate it all. However, there were so many features that we were confused. On the other hand, with the WhaTap solution, we could easily check failures in real time.
We heard that there was a transition to the cloud. Would you please tell us how you used the WhaTap solution during the transition?
Dong-hoon : We have been using WhaTap Application Monitoring for over 5 years. When we switched to Kubernetes this year, we also switched to Kubernetes Monitoring.
Hyun-min: We moved the DB server internally in the second half of this year. At that time, WAS and DB zone failed for a while. There were part of WAS and DB in the KT Cloud, but a failure occurred when the DB was first migrated to AWS.
From the analysis, we found it was a problem caused by too many DB queries. We used the WhaTap solution for tracking. When looking at the log, we can use the function to sort the list by query count. In the same zone, the required time is so short that it doesn't matter. By the way, when the zones are different, we found that the number of queries was high. Because the WhaTap solution can display the number of queries, we could solve the issue by finding which end point had problem.
Dong-hoonThe round trip time (RTT) was short when the DB query and application server were in the same zone. When the DB zone was moved and it became different from the application server’s zone, the latency is increased from 1 ms to 6 ms. In terms of the time for a single query, there is no much difference between 1 ms and 6 ms. However, in case of hundreds of queries, problems arose. Because the required time was short and the resources used for a single query were a few, it was not captured by the DB slow query log and resource usage. If we did not use the WhaTap solution, it was difficult to find the root cause.
Apart from this, we see the WhaTap graph caused by backend interview problems to find failure situations. We can also display various graphs (hit map, memory, etc.) to find which part was the cause of the failure. WhaTap graphs are widely used in the fields of good thought flow and problem solving methodology.
Lastly, would you please feel free to tell us what you want from the WhaTap services?
Dong-hoon: CTO once gave an opinion that he wants to see the entire service status through the WhaTap screen. And I agree with him. For reporting purposes, we want to have a function to observe the communication status of the services or the overall flow. While detecting failures, there is an attempt to search for patterns. It will be better if a function is added to show which patterns can be ignored from the displayed patterns.
Hyun-min : In my case, it will be nice if the error catching technique is improved further during auto scaling. I also want to have the function to pattern repeated notifications. For example, we couldn't save an emoji in our legacy DB at the character setting stage. At that time, an error occurred when user entered an emoji at the application stage. It was not a major error from our point of view. However, it will be better if we can set a notification, “You can set to ignore the part,” by patterning such cases.
Recently, technology-based startups are increasing that are not large enough to have a separate Operation team. As a backend developer of Opensurvey who has been using the WhaTap solution for the last 5 years, would you please advise if you recommend the WhaTap solution?
Dong-hoon: It is often said that the cost of using a monitoring solution is much cheaper than hiring people. Even 3 or 4 years ago, we were curious about the possibility. As the COVID-19 pandemic is getting longer, the market paradigm was totally changed to adopt the monitoring solution. Hiring is very difficult but I want to say that the monitoring solution can solve many problems, especially if you have no skilled Operation team.