“Coalesce has not only been a 3x productivity multiplier for our data team, it has helped us restore trust and enhance collaboration with the business.”
An explosion of data
Challenges
Growing data pipeline complexity impacted reliability of existing assets and slowed delivery of new ones
Company dealt with enormous volumes of data, both operational data and data pouring in from social media platforms running its ads
Inability to scale the data operations to meet the demand of the business
TubeScience deals with huge volumes of data—both its own internal data as well as the firehose of data coming from all the social media platforms where its ads run. When Chief Data Officer Jay Gimple first joined the company, the data team was doing everything manually, which was time-consuming, inefficient, and prone to errors. “Everything was linked and piped in my head and then replicated in the database by good old hand code,” says Jimmy Ched’homme, TubeScience’s Manager of Marketing & Analytics Engineering. There was no documentation, no versioning, no deployment mechanisms. All of this contributed to troubleshooting woes, and made onboarding new team members a daunting task.
In addition, the company’s data pipeline complexity was growing exponentially, impacting the reliability of existing assets and slowing down time to market for new ones. “This is a common affliction of many warehouse initiatives I’ve seen at other companies,” says Gimple. “You start building and get the first few things out pretty quickly, but then you add something else and it becomes increasingly complex. The complexity just grows and slows you down.”
The overburdened data team did its best to fulfill the requests that were coming in from other parts of the business, but it became increasingly difficult to deliver, and there was uncertainty in the data quality. All this led to a lack of trust in the data team across the wider organization. “We had gotten to the point where folks were not putting in data requests for anything because the output wasn’t great,” recalls Darlene Ghorbanian, TubeScience’s Chief Revenue Officer.
And it isn’t just other internal teams that must have confidence in the data being given to them, explains Ghorbanian, but also TubeScience’s clients, who base their sizable ad spends on the effectiveness of the video ads the company produces for them. “We’re a data-driven team, and every pitch we take to our clients incorporates data to support the ask,” says Ghorbanian. Employees who manage clients directly would spend several hours compiling numbers on ad performance, careful not to misrepresent anything. “Our clients are also very data- and performance-driven. They often have these numbers in their head so they’ll know if you’re wrong, and that damages trust,” she says.
Considering the costs
Solution
All data in one place with Snowflake
Automate data transformation with Coalesce to enable speed and scale—and lower TCO
Build early dashboards to analyze problems quickly and re-establish trust
Gimple began looking for the right transformation tool to help his team overcome their data challenges. TubeScience had already built out its data and analytics capability on Snowflake, so choosing a solution that was built for Snowflake made a lot of sense. But he was also interested in Coalesce’s long list of capabilities that promised to help untangle their complex data pipelines and scale more quickly. “Faster builds and iterations, rapid prototyping, enabling faster support and troubleshooting, really great documentation, and table and column lineage are key to being able to identify what may be wrong in the pipeline,” he says. “Or to help other people understand how the data is flowing.”
But for Gimple, the key advantage of Coalesce over competing platforms was the one key factor he always considers when making a decision about a vendor: the total cost of ownership.
“There’s certainly other ways to build out a pipeline in Snowflake, but with the cost of the pipeline over the next five years, what does that look like?” Gimple says. “If the solution requires highly skilled professionals, your staffing costs increase. If the complexity of your pipeline grows exponentially as you build onto it, you need to hire an army, both to maintain it and to get new requests shipped. So these all add to the costs, and for me it was important to find something that was going to be a multiplier for the team.”
In hopes of improving the lack of trust in the data team, once Coalesce was brought onboard, Gimple and Ched’homme focused on building initial dashboards and getting them in front of colleagues as quickly as possible. “When you present results,” Gimple notes, “you frequently encounter feedback such as ‘This number doesn’t seem accurate.’ The real test often lies in analyzing whether it’s factual or merely a perception. With Coalesce, the team could efficiently carry out this analysis. If any discrepancies arose, they were rapidly addressed. When facing a ‘feeling,’ they showcased how the data substantiated their findings. Either way, being able to address concerns promptly is a surefire way to build trust.”
Tripled productivity, instant insights
Results
3x multiplier for the data team: with Coalesce, one data engineer can do the work of three engineers
Ad performance evaluations that took hours can now be done in an instant
Changed the perception of the data team across the larger organization
Using Coalesce has resulted in remarkable productivity gains for the data team. “Since we introduced Coalesce, I can now do the work of three engineers,” says Ched’homme. “Just knowing where to troubleshoot, what to troubleshoot, how quickly I can propagate changes, add fields or remove them, understand where it went wrong, implement tests … if we had not had Coalesce, we probably would’ve had to hire a few more people to do the job I’m doing today.” This alone means the company is saving on the salaries of two data engineers.
Coalesce’s GUI enables anyone to quickly see the data pipelines, how they’re connected, and where the data flows. “Being able to see the pipeline as a kind of ‘3D object’ is something that I particularly love,” Ched’homme says, as the visual interface makes it simple to troubleshoot and identify and track errors. “Sometimes you find yourself spending more time trying to understand where things have gone wrong than actually working on developing the pipeline. But that isn’t the case with Coalesce. Instead, it’s easy to identify the problem, understand what’s wrong, change it quickly, propagate that change, deploy, and try again—it’s so much more efficient.”
Coalesce allowed Gimple to finally tackle the core problem he needed to solve when he first came onboard, which was to get a handle on the company’s exploding data pipeline complexity. “With Coalesce, we can continue to build but have the confidence that the complexity is growing more linear instead of exponentially, because the platform is focused on reducing the complexity of building out the pipeline,” he says. In addition, Coalesce greatly lessens the time needed to onboard a new engineer. “If you have an organically grown data pipeline, ramp-up time for a new engineer could be four to six months. In Coalesce, we can ramp somebody up in a week or two and start getting value from them,” says Gimple.
The initial dashboards Gimple’s team built out truly enabled their data consumers across the organization, and today people are putting in data requests again because they get done so quickly. “It’s great because we are able to dive in, triage those, and get responses back relatively quickly because of the pipeline,” says Gimple, “I think that really helped increase trust.” Ghorbanian agrees: “The perception of the data team has changed greatly internally, and folks are really excited.” What’s important is not just how fast the results are, but that what Gimple and Ched’homme are doing with Coalesce helps these users to truly understand the data behind the dashboards.
Now that TubeScience uses Coalesce, Ghorbanian also sees an enormous benefit for those employees who had been struggling to pull accurate campaign performance metrics before client meetings. “I’ve heard some of my colleagues say, ‘I feel like I can breathe.’” Before having access to the PowerBI dashboard created by the data team, they had to manually manipulate data throughout the entire cycle—from the data team to the end user. “So you’d have to QC your very manual and brittle process,” she says. “Now we just click on the PowerBI and we have that information within five seconds. It’s click, click, bang and you have the number.”
