PODCAST: Making AI Real
Episode 4: Unlocking the Value of Enterprise AI with Data Engineering Capabilities
Listening time: 12 minutes
Unlocking the Value of Enterprise AI with Data Engineering Capabilities
In this episode of the AI to Impact Podcast, host Pavan Kumar speaks to Prinkan Pal about the evolution of data engineering and ML-operations from a closed team into a tech consulting unit. They discuss how the data engineering team is instrumental in easing collaboration between analysts, data scientists and ML engineers to build enterprise AI solutions. While some companies prefer plug and play solutions, others prefer to customize their platforms by integrating with open-source tech stacks. The experts also remark upon the rise in the demand for tech platforms that can scale seamlessly to handle volumes of data and generate actionable insights in no-time across industries. Tune in to the podcast to know more about the evolving industry and how new technologies are transforming the enterprise AI landscape.
‘There is a lot of demand for data engineering teams to enable platforms that eases collaboration between analysts, data scientists, and ML-engineers to build what we call as enterprise AI solutions.’
Pavan: Hi, everyone. Welcome back to our podcast – AI to Impact, BRIDGEi2i, a podcast on AI for digital enterprise. As you know, we bring to you the point of view and commentaries from remarkable thought leaders and industry experts from the AI and analytics domain, an understanding of how to make AI real for any organization. I’m your host – Pawan Kumar. I lead the Intelligent Automation practice focusing on application development and platforms. My team has a mix of cloud architects, project managers, designers, and full-stack developers. Today, when data is generated from different sources like text, image, videos, sensors, and so on, extracting actionable intelligence from these massive data sets is an art and a science. Enterprises are still learning to master this art. Today in this podcast, we have Prinkan Pal, who is heading data engineering and ML-ops practice at BRIDGEi2i. He has driven some successful projects in financial services, CPG, and enterprise technology. Can you tell me a little bit more about your work at BRIDGEi2i, Prinkan?
Prinkan: So I lead the data engineering and ML ops practice, like you mentioned, at BRIDGEi2i. My team primarily comprises data engineers, ML-engineers, full-stack developers and solution architects primarily focusing towards cloud. We do both, you know, technology consulting and execution. In other words, we form the tech backbone of data and analytics platforms for our clients who are at different stages of their digital transformation journey.
Pavan: Great, Prinkan. On the recent events arising from the global pandemic, as a reminder for us that change is the only constant in life, in business. Now, I think it has forced a lot of enterprises to fasten their digital transformation journeys so that we’re better equipped to kind of respond. And the core of digital enterprises is the ability to leverage data and deliver actionable insights through AI. I think companies are really being forced to find ways to kind of turbocharge these operations. How do you see data engineering fit well into the spectrum of changes?
Prinkan: Yeah, I think, you know, besides the fact that data engineering encompasses data operations, there is a lot of demand for data engineering teams to enable platforms that eases collaboration between analysts, data scientists, and ML engineers to build what we call as enterprise AI solutions. So the industry has realized that analytics sophistication through AI is impossible without data modernization, and hence, in the recent past, a lot of focus has been towards: AI-driven data cleansing approaches to ensure quality rich data for analytics, or like, you know, data catalogues to tag metadata with business glossary and enable search-based discovery, on the other hand, features tools to manage a repository of reusable data attributes in the process of model building, analytics sandbox as a governed environment for data scientists to experiment, train, test and evaluate their models. Also, ML-ops or machine learning operationalization approaches to bridge the gap between sandbox and production pipelines, while ensuring model lifecycle management from raw data to predictive, as well as prescriptive, insights integration.
Pavan: I think Prinkan, you’ve touched upon the concept of experimenting with ML technologies. That’s one part to it. I think, the difficult question today is integrating models into business applications and processes. And we’ve touched upon ML ops as well. How can companies kind of operationalize these ML technologies?
Prinkan: Yeah, I think one needs to really understand the nuances of their business processes, and the needs of analytics in context to the same. There are numerous tools and ML platforms that can help streamline ML or AI ops, but the question is what is right for you?
So in my experience, I have seen certain enterprises that have brought certain ready-made products, while we have clients for whom we have built a customized platform by integrated open-source tech stack. Now, both approaches have its own pros and cons, and hence needs very thorough evaluation and quick POCs to help make an informed decision.
Pavan: That’s an important point, Prinkan and I think trying to kind of reuse existing open source solutions, more of off the shelf products is an important aspect to it. And in that, have you seen any shift in the trend in the last six months?
Prinkan: Yeah, I think this entire situation of COVID has brought in a lot of digital transformation initiatives across industries. A couple of them that is very remarkable, at least the work that I’m doing across different clients or conversations I’m having, let’s say one in the consumer packaged goods industry that I see is a shift from in-store shopping to more e-commerce platforms and, you know, hence, the investments for customer experience initiatives on digital platforms have significantly gone high.
The other one being increased needs of door-to-door delivery has resulted in the demand of more sophisticated systems of supply chain management, to you know, to track, to predict more accurately of the demand etc. While on the financial services side, I have seen, you know, a rise in the no-touch banking kind of systems, where mobile banking or third party payment integrators, applications, communicators, all these are coming into play where a lot of data is getting generated, which needs to be processed and fed back to these systems to make them more intelligent. So, some of those are, you know, very relevant in the context to the industry today.
Pavan: Yeah, I think there’s quite a variety of industries, you touched upon. Any specific client interaction you felt Prinkan recently, maybe you can share some interesting use cases on that?
Prinkan: Yeah, one of them, I think, which we are currently working on is with a global electronics manufacturing company, where they are trying to implement a customer 360 platform to enable purpose-based search or recommendations, more than a feature basis. So, you know, what we are really trying to do is by extracting product features, and extracting customer needs/intent from, from a conversation and create a knowledge graph, which can be used to search in real-time. So basically, what we do is a lot of the, you know, interaction that happens between the customer and the e-commerce portal or the you know, marketplace, that allows us to prove certain questions to the customer and get certain intent of what kind of product or what is the context for which the product is being bought, etc, based on which we can generate more of purpose-driven recommendations rather than feature-driven filtering of the products. So that is one of the use cases that uses a lot of graph algorithms and graph data.
Pavan: Got it. Prinkan, I think a lot of things has to be based on people. You have been with BRIDGEi2i for like six years now. You built the data engineering team at BRIDGEi2i. What is it to kind of run a team of that type?
Prinkan: Yeah, I think it has been a pretty adventurous journey, adapting to the changing and the growing needs of the customer and streamlining the team for growth phase has been very, very challenging. My learning from this is the best mechanism to grow faster is to constantly democratize departments and teams with tech capabilities that we have learned very well. So in BRIDGEi2i, we have the centres of excellence, run by the experts from my team that helps democratizing technology across the organization. How is it on your side, Pavan? I wanted to understand, you are also leading the Intelligent Automation practice, so how do things work there?
Pavan: Yeah, it’s been exciting to shape up the team as well Prinkan. We brought in some few structures to the team, we want to kind of bring in, let’s say, the right process from RFP to delivery to hypercare, introducing new technologies as well. We’re kind of bringing in Svelte for UI development and Kedro for Code pipelining, and all of that. So I think in some sense, the new technologies kind of shaping up, I’m sure, I think it’s helping you differentiate as well with new technologies or the data engineering side as well. What is your view specifically on tech?
Prinkan: Yeah, we are constantly experimenting with a lot of different technologies, especially the open-source stacks ones primarily focused around data storage, compute, and operationalization of AI algorithms. A dedicated team constantly focuses on stress testing these technologies and also trying to hack some of those to understand the loopholes, etc. So one of the key value-add to our customers has been our ability to provide cost-optimal technology solutions by maximizing the use of open stack without compromising enterprise standards.
Pavan: Okay. I think one of the key trends that we have seen is like customers used to say like, it takes five years to execute a roadmap and maybe three years for a digital transformation, maybe two and a half to build it. I think now it has changed everything. Now they want everything in weeks, sometimes even days. Again, it’s not a matter of choice that they would want to do it that way and how do you think the whole DE space is evolving in future?
Prinkan: Yeah, like, you know, everyone keeps saying data is the new oil and everyone needs data. So, I think there has to be a paradigm shift that draws from modern distributed architecture considering domains as a first-class concern, applying platform thinking to create self-serve data infrastructure, and treating data as a product. So decentralizing data teams and not having only a centralized team that should do everything and democratizing people with the abilities to process and generate insights from data. Whereas having probably a kind of a centre of excellence, which we also do in our organization, trying to monitor if best practices and standards are being followed, I think would help a lot in fast pacing the course of events that is required to, you know, generate actionable insights from raw data.
Pavan: Really fascinating discussion, Prinkan. I think, all said about the professional hustle we have been discussing about. Curious to know, like, what keeps you busy apart from data, lakes and technologies, what we just discussed?
Prinkan: So I spend quite a lot of time reading books of different kinds as it gives me you know, different perspectives. Once a week, I teach math and computers in an NGO. Apart from that, I enjoy cooking and watching sci-fi movies.
Pavan: Prinkan, loved the conversation. Thanks for making the time for this interaction today.
Prinkan: Thanks for having me, Pavan.
Pavan: Thank you so much for listening to this episode of AI to Impact podcast. It was a great discussion with Prinakn, Data Engineering and ML-Ops Practice Head at BRIDGEi2i Analytics. A lot of valuable insights on how the industry is changing, the adoption of new technologies, and human side of technologists. If you found this interesting, do subscribe to our podcast and leave us a review in the section. We would love to know your thoughts and if you would like to kind of know more about any specific topic, please write to us in the comment section. Signing off. Stay home, stay safe.
With new data formats, evolving data ecosystems and AI technologies becoming more mainstream, the intersection of Data Science and AI-powered interaction systems present a whole new gamut of opportunities for digital enterprises. In our latest AI to Impact podcast series, Making AI Real, we interact with several thought leaders, BRIDGEi2i business heads, domain experts, and reputed AI and analytics leaders to learn more about the ground realities that further AI innovations. Stay tuned to know more!
Pavan Kumar – Director, Technology – BRIDGEi2i Analytics Solutions
Pavan leads the Engineering & Application Development efforts at BRIDGEi2i. He has over 2 decades of experience designing and developing software solutions in the telecom and retail domain. He has created products at scale related to Personalization, eCommerce analytics, SKU Optimizations, and Image search in his earlier roles. Pavan is an alumnus of the College of Engineering, Guindy.
Prinkan Pal – Director, DE & ML Ops Practice
Prinkan heads the Data Engg & ML Ops Practice and is responsible for talent development, innovation, design, and delivery of AI solutions for the digital enterprise. He is considered the SPOC for all Data Engineering, Visualization, and Algorithm Operationalization needs of BRIDGEi2i. Prinkan has a Bachelor’s Degree in Electronics and Communication Engineering from Visvesvaraya Technological University (VTU). He is a Member of the Institute of Engineering and Technology (MIET), UK.