The takeaways, in brief
- AI changed the traffic shape. More sessions, harder to separate good from bad — so Databricks leans on its own agents to filter traffic and apply policy.
- Shadow IT is a discovery problem. The team is mapping which apps are used for what, validating use cases, and building policy without blocking engineers — a standing item in every QBR.
- Layer the controls. Continuous authentication, certificate-based passwordless, automated cert rotation in infrastructure-as-code, and a reduced L2 blast radius — not a single security vector.
Kyle Dimmick runs the corporate-side network engineering team at Databricks, the company at the center of the AI and data revolution — everything from corporate offices to cloud connectivity. He joined Okta’s Chris Carlson on the SecureW2 × Myriad 360 customer panel at The Battery in San Francisco, a live Q&A moderated by SecureW2 co-founder Bert Kashyap. We pulled the network thread out of his answers.
Databricks sits at the center of the AI and data revolution. From a networking seat, how is the attack surface evolving?
From a networking perspective, we’ve seen a lot more traffic when it comes to AI — a lot more sessions being generated, and it’s a lot harder to filter through good versus bad. So we’ve seen a really strong reliance on using our own agents to filter through that traffic and policy.
How much of the shadow-IT sprawl do you have to own, and how are you handling it?
The goal is never to hinder productivity for engineers or those trying to find new tools to do their job more efficiently. But determining sanctioned versus unsanctioned, and policies for unsanctioned applications — what kind of information you can provide in there — those are all things that are very much top of mind right now.
We’re in discovery phase, trying to figure out what apps are being used for what. Do they have valid use cases, and how do we build policies around that while enabling people to get their work done? There’s plenty of support from the CISO and the other executive leaders — it comes up in every QBR, and it’s something we have to demonstrate progress on constantly.
What does layered identity and authentication look like for you in practice?
Something we’re trying to do is continuous authentication. Not everything supports it, but when we can have that extra layer — machine auth or user auth happening in the background — then add that on top of IP ACLs and firewall rules. Layer your security and don’t just rely on one vector.
Newer applications support passwordless — you can leverage IDP for Okta Verify FastPass and continuous authentication with certificates. But there are still legacy systems out there that have passwords, and it’s a daily battle, either phasing them out or rebuilding them. Trying to remove the need for the password in the environment is just a constant battle.
We can take all these logs from the different aspects we use for network protection and pipe those into the Databricks product, and use agents on top of that to log, analyze, and correlate events.
— Kyle Dimmick, Databricks
Where does quantum readiness sit on the network side?
Focusing on the network side: certificate rotation. Being able to rotate certificates, building things with infrastructure-as-code so you can automate that process and not do it manually — and then figuring out what is living where. A lot of enterprises don’t know where things are, or they have random one-offs that need to be rotated manually. Integrating that into a simple platform you can manage through code is important. Following industry best trends and certificate rotation, you have less to worry about with post-quantum.
The SecureW2 read
Dimmick’s playbook is the cert-based identity story told from the network side: automate certificate rotation in code, know where every credential lives, and treat continuous device trust — not a single firewall rule — as the control plane. When the L2 blast radius shrinks and identity travels with the device, the AI-scale traffic problem becomes a correlation problem, not a perimeter problem.
Quotes are drawn from the SecureW2 × Myriad 360 customer panel at The Battery, San Francisco. Lightly edited for length and clarity. Used here in a design mockup.
