1. Introduction

Whenever user goes online either on a website or App, Ad Server or SSP gets all the available information about user and website/App, send the same to all Demand Partner Server.

Most of publishers work with multiple Supply partners and have restriction of getting an Ad within 100 ms to 300 ms. SSP performs some optimization and send the information to all connected Demand partner server. Demand partner server need to respond within allowed period which can be 100ms to 200ms.

In the entire work flow if any involved entity Demand partner or SSP couldn’t respond within allowed time lose the opportunity of showing Ad to the user. This situation is known as Timeout.

Hence, Timeout is a lost revenue opportunity. It’s very important to regularly keep check on Timeout, analyze the same and notify involved parties with troubleshooting information.

Challenge is that every SSP or AdServer have multiple Data center and Timeout can be a problem on a specific Data center with a specific Demand Partner. So, we need a method to analyze Timeout at every Data center and troubleshoot the same with Demand partner having timeout issue.

It’s very often said in programmatic world that Lost impression is a lost revenue. Throughout the Article, we would be using few keywords which are as below.

SSP – A supply-side platform is a piece of software used to sell advertising in an automated fashion. SSPs are most often used by online publishers to help them sell display, video, and mobile ads. [1]

DSP – A system that allows buyers of digital advertising inventory to manage multiple ad exchange and data exchange accounts through one interface., Real-time bidding for displaying online advertising takes place within the ad exchanges, and by utilizing a DSP, marketers can manage their bids for the banners and the pricing for the data that they are layering on to target their audiences. Much like Paid Search, using DSPs allows users to optimize based on set Key Performance Indicators such as effective Cost per Click (eCPC), and effective Cost per Action (eCPA). [2]

OpenRTB Specification - The mission of the OpenRTB project is to spur growth in Real-Time Bidding (RTB) marketplaces by providing open industry standards for communication between buyers of advertising and sellers of publisher inventory. There are several aspects to these standards including but not limited to the actual real-time bidding protocol, information taxonomies, offline configuration synchronization, and many more. This document specifies a standard for the Real-Time Bidding Interface.

OpenRTB has multiple version of real time bidding specifications like Version 2.0, 2.1,2.2, 2.3, 2.5. [8]

Impression - An impression (in the context of online advertising) is when an ad is fetched from its source, and is countable. Whether the ad is clicked is not considered. Each time an ad is fetched, it is counted as one impression. [3]

2. Research Method

If we look at the Real-time bidding ecosystem, Timeout can happen because of two Reason.

1. Due to Latency in Network

2. Due to Latency in DSP System/Server

If we analyze the Network Time between SSP Server and DSP Server, we can find the Network travel/Connection time. If that’s high, it means there is problem with Network.

In the case Network latency is high, we can analyze the time between each gateway by looking at the path that was used to communicate from SSP Server to DSP Server, to find the route/gateway which is exactly causing the Network latency. As it could be due to issue in ISP provider of SSP or ISP provider of DSP.

We can use traceroute [6] or tracert [7] like tools which Internet Control Message Protocol (ICMP) Echo Request messages to the destination with incrementally increasing Time to Live (TTL) field values. The path displayed is the list of near-side router interfaces of the routers in the path between a source host and a destination.

If there is No Latency in Network, Then We need to analyze the full time taken by DSP to respond.

As it might be possible that, DSP system is taking a lot of time to process an Ad Request.

DSP generally respond with either an empty response or a valid Ad response. Empty response means that DSP is not able to show Ad for the requested Ad request.

Look at the time taken by DSP for both kind of responses, we can find patterns as below.

• Time taken by DSP for both empty and valid response takes more time than allowed.

• Time taken by DSP for empty response is low but Its very high for responding with a valid Ad response

In below Diagram, the flow of data from the time User get online till the Ad is served to the user is demonstrated.

Below is the sequence of events happen to show an Ad on a digital platform.

1. User goes online (Either on a website or App or smart device)

2. Ad Server gets all the information from the device and send the same to SSP

3. SSP send Ad requests to all connected DSPs

4. DSP process the request and return with an empty or valid Ad response

5. SSP only entertain Ad response from DSPs who can bid within the allowed time frame otherwise they are considered timed out.

6. SSP perform internal auction, if applicable

7. SSP send Ad response back to Ad Server

8. Ad Server also has time restriction and entertain response from a SSP who can bid within the time constrain

9. Ad server renders the Ad on the Digital content (website or App)

Flow Diagram of Ad request:

Figure 1

3. Analysis and Implementation

Every SSP or AdServer can have multiple Data Center. Every data centers can have different timeout restriction. Every DSP also have respective Data Centers to avoid Latency.

We can apply the research method to analyze timeout root cause from a specific data center of SSP to respective DSP endpoint or Data Center.

We can create a service to look at timeout stats periodically and analyze timeout using above method for the Data Center which has high latency.

Let’s define the timeout for ever Data center and use traceroute to find if Network has an issue. We should look at every gateway and highlight that path which is causing high latency.

Send an email with the Timeout stats with traceroute [6] samples from SSP server to DSP endpoints highlighting the gateway which is causing high latency.

Traceroute Example:

Below highlighted gateways have high latency. This information can be used to notify the ISP provider to fix the network routing.

If there is no Network Latency, then we should look at response time of DSP as mentioned in Research method.

We can do the same by using curl command. We can use curl command to send Ad request to DSP having high timeout from the Server at Datacenter which has high timeout.

Analyze the various kind of responses received from DSP as mentioned in research method to find the pattern and highlight the ones which is causing high timeout.

Gather all those information for every DSP having high timeout and send the same in an email.

Implementation Flow Diagram:

Figure 2

Pseudo Code Implementation using php:

Assumption:

We are using Linux environment. All the DSP information is stored in MySQL.

We already have keyless SSH access to servers which would be used to perform troubleshooting and Analysis.

1: Create a Service which runs on regular interval like hourly and find the DSP which are having high Timeout and execute campaignToEmail.php by passing respective DSP or campaign Id which has high timeout.

2: Create a config file to store IPs of servers for each data centre which is available to perform troubleshooting. Name the file as ServerConfig.php

Method Usage and Explanation:

Below method takes the Data Center Id as argument and return the list of servers for the specific Data center which can be used for troubleshooting.

Point to be Noted: We can have 100s or 1000s of server at a Data center, we should limit ourselves to specific set of servers to use for troubleshooting and analysis.

ServerConfig.php code snippet:

3: Create a file CampaignToEmail.php which will take DSP Id or name as input and analyse Timeout for the same at all configured data centre and store the information in a html file.

Function Explanation:

It takes 3 arguments.

$Task – Task name, TOD is the task name used for Timeout Detail

$campaignID – Id of DSP Campaign or Account

$CampaignName – Name of DSP Campaign or Account

It checks if user has permission to run the task. It executes CampaignTo.php for the requested task (which would TOD) and store the output in a temporary file.

It uses another helper method SendMail.php to send an email the respective DSP partner with the content of temporary file.

Temporary file has all information about the Timeout Analysis.

Pseudo Code snippet:

4:Create a file CampaignTO.php. It will create Instance of class Campaign (should be declared in Campaign.php) and call the main function Campaign->Timeout.

Code Snippet:

5: Create a file called Campaign.php which will store all information about a DSP like endpoints, threshold timeout for each data center.

It will have multiple methods to extract all related information about DSP.

Below are few variables which would be used at below sample method.

Campaign.php will perform two types of Analysis.

1. Traceroute Analysis

2. DSP Response Time Analysis:

5.1: Traceroute Analysis

Below is the main method which can analyze traceroute [6] results executed from SSP server for the Demand Partner API URL.

Arguments to the Function:

5.2: DSP Response Time Analysis:

There is a possibility that there is no issue in Network. Traceroute analysis will show good results in that case.

So, now we should send sample request to DSP API url.

Please note that most of DSPs use optimization and don’t bid on every ad request. We need to send bid requests to get two types of Responses.

1. DSP bided with no or empty response

This will tell us the time, it took for a DSP when they choose not to bid. We should try to get sample of multiple such responses. Most of the time DSP should be able to decide early not to bid and response time should be relatively low.

2. DSP bided with an Ad response

This will tell us the time, it took for a DSP to process the Ad request and apply all internal decision rules and return an eligible Ad response. If there is a latency in a DSP engine or bottle neck situation, time will be high.

In the case we see the time in both types of response is very high, then it direct DSP to check their internal setup or network.

Please refer Open RTB 2.5 [7.0] specification to get sample OpenRTB Ad Requests.

Below is few sample curl call to send Bid requests to a DSP API URL.

Curl Sample to send an Ad request to a DSP:

Curl Sample to send an Ad request to a DSP through a Remote Server:

4. Conclusions

By Analyzing Network Latency and Response Time of DSP Server, we can troubleshoot and find root cause of Timeout in a Real Time Bidding System. If there is an issue with Network involving SSP’s ISP provided or same for the DSP’s ISP provider, they can proactively fix the problem. SSP or DSP can also use this information to perform Network peering or updating their DNS lookup if needed.

We can also find out if issue is within the DSP System and same can be notified to DSPs regularly through Sample Ad response time. They can use that information to fix the problem in their system.

Regularly monitoring, analyzing Timeout issue in real time bidding system can avoid loss of impression and hence can avoid loss of revenue.