Afleveringen
-
InfluxDB 3.0 Rewrite
InfluxDB, a time series database, underwent a major rewrite to create InfluxDB 3.0, also known as IOx. The decision to rewrite the database was driven by the need for strict control over memory management and high performance. The project started as a research endeavor and gradually gained traction within the company. The team decided to build around projects under the Apache Foundation, such as Apache Arrow and Apache Data Fusion. In April 2022, InfluxDB 3.0 was officially announced, aiming to improve performance, scalability, and cost-effectiveness for users.
IOx Database Engine
The new database engine, IOx, is designed to handle various types of observability and monitoring data, including metrics, traces, and logs. It aims to provide a single store for all these signals, eliminating the need for separate databases. However, querying the data efficiently is still a challenge that the team is working on. The goal is to make IOx the go-to solution for storing and querying observational data, not only for server infrastructure monitoring but also for sensor data use cases.
Challenges and Considerations
Working with logs, tracing, and structured events in time series databases poses challenges. The dynamic and inconsistent nature of schemas in logs and tracing use cases can make extracting structured fields difficult. Time series databases also have limitations in handling tracing front ends and require an index to map trace IDs to individual traces. While metrics, logs, and traces are the gold standard for observability, there is room for improvement in terms of usability and performance.
Flux and Data Fusion
Flux, a scripting language developed for InfluxDB 2.0, addresses user requests for more complex query logic and integration with third-party systems. InfluxDB 3.0 incorporates a parser in Rust to translate SQL queries into a Data Fusion query plan, benefiting from the performance optimizations of Data Fusion. However, bringing Flux to InfluxDB 3.0 proved challenging due to the large surface area of Flux and limited time and resources. Updating the Flux engine to use the 3.0 native API could potentially resolve these issues.
InfluxDB Development and Open Source Licensing
InfluxData is focused on improving the core query engine of InfluxDB and enhancing its capabilities and performance. They have created a separate community fork of Flux to allow collaboration on its development. Paul Dix, the co-founder, believes that true open source should be about freedom and expresses his intention to keep InfluxDB 3 as a permissively licensed project. He discusses the recent license change by HashiCorp and the growing distrust in the developer community towards VC-backed open source projects. Putting InfluxDB into a foundation may not be feasible due to the lack of multiple contributors.
-
Here are 5 key takeaways from this episode that you don't want to miss:
1️⃣ The People Problem: Laura Santamaria raises an important concern about verifying AI-generated outputs and tackling the challenge of the "people problem" in AI development.
2️⃣ Verifying Data Authenticity: JJ discusses the challenge of proving that a data blob originated from a specific model and how this issue is being addressed by companies like IBM through pile cleaning and legal penalties.
3️⃣ AI Misconceptions: We debunk some common misconceptions about AI, including the belief that it is an all-knowing fact machine.
4️⃣ Trusted AI: IBM's approach to building trusted models, with dedicated engineers responsible for cleaning and verifying data, is explained. Plus, we discover IBM's partnerships with Hugging Face to leverage the open-source ecosystem.
5️⃣ The Impact of AI: We delve into the potential positive and negative implications of AI, and how the rapid advancement of this technology presents challenges with trust and validation.
💡 Fun Fact: Did you know that 95% of open-source language models are trained on a data set called "the pile," which contains pirated and copyrighted material? Discover why this has implications for copyright and patent laws!
As always, the conversation in this episode is engaging and eye-opening. JJ Asghar provides insightful perspectives and sheds light on the future of AI development. Don't miss out on the valuable information shared!
Questions We Covered
1. How can the problem of untrusted data in AI models be effectively addressed?
2. Should companies like OpenAI and Microsoft be required to provide their data sets for verification purposes? Why or why not?
3. What are the potential risks and challenges associated with using AI technology without proper regulation?
4. Should AI creations be eligible for copyright protection? Why or why not?
5. How can we ensure the accuracy and trustworthiness of AI-generated data, especially when it comes to extracting information from sources like PDFs?
6. What are some potential positive impacts of AI technology, and how can we maximize its benefits while minimizing its negative implications?
7. How can the rapid advancement of AI technology be balanced with the need for trust and validation?
8. In what ways do copyright and patent laws need to evolve to accommodate AI technology?
9. What are the implications of China having its own set of laws and approaches to technology that may differ from other countries?
10. How can individuals navigate and better understand the AI space in order to make informed decisions and contributions? -
Zijn er afleveringen die ontbreken?
-
We're back with an exciting new episode of Cloud Native Compass, and this time we're diving deep into the captivating world of eBPF. Join Laura Santamaria, David Flanagan, and special guest Liz Rice as they unravel the mysteries and explore the incredible potential of this powerful technology!
In this episode, you will learn:
1. The two parts of eBPF: Discover the kernel program and the user space interaction that make up the magic of eBPF.
2. Programming with eBPF: Explore the different options for writing eBPF programs, from bytecode form to higher-level languages like Rust.
3. Compilers and SDKs: Learn which compilers, like clang GCC and the Rust compiler, support eBPF bytecode and how SDKs can make your interaction with eBPF programs easier.
4. The Evolution of Packet Filtering: Trace the history of packet filtering, from its humble beginnings to the powerful and versatile capabilities of eBPF.
5. The Widespread Adoption: Uncover the sudden rise in eBPF's popularity, its impact on observability and performance, and the role it plays in modern networking.
Now, for a fascinating fun fact from the episode: Did you know that eBPF is now considered Turing complete? That's right! With its combination of features, eBPF has surpassed its humble start as a packet filtering tool and has become a full-fledged technology powerhouse.Creators & Guests
Laura Santamaria - Host David Flanagan - Host Liz Rice - Guest
(01:46) - What is eBPF?(06:45) - The Rise of eBPF(09:40) - Why is eBPF Interesting?(16:00) - Who's using eBPF?(19:20) - eBPF for Developers(24:00) - Troubleshooting eBPF(27:11) - Future of eBPF -
Curious about Istio's new deployment mechanism, Ambient Mesh? It allows you to use Istio service mesh without relying on sidecar proxies, which brings a bunch of improvements that Marino and Matt share throughout this episode; as well as a ton of deep dive technical implementation details.
Creators & Guests
Laura Santamaria - Host David Flanagan - Host Marino Wijay - Guest Matt Turner - Guest
(00:00) - - Introductions(01:50) - - What Ambient Mesh?(04:15) - - Why Ambient Mesh?(18:20) - - Waypoint Proxy(25:00) - - Trade Offs(34:20) - - Why Not eBPF?(39:50) - - Istio Graduation! -
In this episode, hosts David and Laura, sit down with Laslo Fogas; a self proclaimed WebAssembly sceptic. They discuss the future of Cloud Native and improving the broken developer experience.
Creators & Guests
Laura Santamaria - Host David Flanagan - Host Laszlo Fogas - Guest
-
In this episode of the Cloud Native Compass, host David Flanagan interviews Natan from Wix Engineering about event-driven architectures.
Natan shares his experience as a software engineer for almost 20 years and how working at Wix has improved his engineering skills. Wix has a powerful website building platform that has enabled people with different skill levels to build websites. They have expanded their reach from self-creators to agencies and web professionals and created a whole ecosystem platform. Wix has around 2,500 microservices in production, even more added every week, and they have a lot of visitors, around 1 billion unique visitors every month, which gives more than 500 billion HTTP requests per day and 70 billion Kafka events produced every day.
Let's learn how Natan and Wix build for success at some pretty stagger numbers.
Creators & Guests
David Flanagan - Host Natan Silnitsky - Guest
(00:00) - Introduction(02:25) - The Scale of Wix(08:50) - When & Why Event Driven Architectures(14:45) - Service Mesh(19:30) - Dev & Ops(27:15) - Schema Evolution & Versioning(34:00) - Introducing New Tools(37:15) - Cost Optimisation(44:44) - Plugs(00:00) - Chapter 10
Linkshttps://www.natansil.com/
https://www.wix.engineering/ -
I interview Marc Boorshtein, the CTO of Tremolo Security, an open-source identity management company that focuses on authentication, authorization, identity, and automation. Marc explains that their most popular tool is Open Unison, which allows users to log in to their Kubernetes clusters with whatever authentication system they have, such as LDAP, AD, Okta, or Azure AD. Open Unison also provides secure access to the dashboard and integrates with other cluster management applications.
Next up we shift over to the issue of certificate revocation in Kubernetes. Marc explains that Kubernetes doesn't know how to handle certificate revocation, which can be a security risk if a certificate is leaked or an employee leaves the company. He recommends using OpenID Connect or impersonation to access the cluster instead of relying on certificates. Marc also discusses the default time to live on service account tokens issued by the Kubernetes cluster and the importance of not using service account tokens when talking to clusters
This episode provides insights into the challenges of identity management with Kubernetes and strives to help you improve the security of your Kubernetes clusters.
-
In this episode, Rachel shares her journey into tech and how she ended up in the Kubernetes space. She did not have a traditional IT background, but she was always interested in computers and programming. As soon as she discovered the DevOps philosophy and movement, she knew that it was the right fit for her. She went down the rabbit hole of learning how to use Docker, Ansible, Chef, and Kubernetes, and eventually landed a job at Fairwinds, a Kubernetes-centric company.
Rachel discusses the challenges that come with adopting Kubernetes, such as the steep learning curve and the knowledge gap. There are many unknowns if you are not in the Kubernetes space and it can be overwhelming to configure to produce a valuable platform for your teams.
Rachel's journey into the tech space and her experience with Kubernetes provide valuable insights into the challenges and benefits of adopting and migrating to Kubernetes.
Rachel Sweeney is the tech lead at Built Technologies for their migration to Kubernetes. Prior to that she was a tech lead at Fairwinds, a Kubernetes SaaS and consulting company, and before that she worked at the Pew Research Center creating their Kubernetes cluster and migrating workloads to it.
She has been a speaker and panelist at various conferences and events ranging from DevOpsDays Philly to Container Journal, and also wrote a chapter for the O’Reilly title “97 Things Every Cloud Engineer Should Know: Collective Wisdom from the Experts”.
Rachel loves traveling, culture, meeting new people, networking, and helping others grow. Feel free to reach out on LinkedIn with a message and connect!
Creators & Guests
David Flanagan - Host Rachel Sweeney - Guest
-
In this episode, David interviews Frank, a staff engineer at Personio, a big HR company based out of Germany. Frank has been in the industry for close to 15 years and has worked with various programming languages such as PHP, TypeScript, Kotlin, Java, Go, and Rust.
Frank's interest in event-driven architecture started when he worked on a project for Schiphol airport. The website needed to ingest flight information as close to real-time as possible, and the only real way to do that for the scale they were at was through an event-based integration. Frank believes that event-driven architecture is the only way to do things for some projects.
David Flanagan - Host Frank de Jonge - Guest
People
Creators & Guests
Chapters
(00:00) - Introductions(01:45) - The Knowledge of Others(06:35) - A Need for Event-Driven Architectures(10:45) - When to Event Source(19:15) - What is Event Sourcing(24:50) - What is CQRS(27:40) - Crossover of Microservices and Event-Driven(36:45) - Challenges of Event Sourcing(41:40) - Exciting Futures
Links- Frank's Twitter - https://twitter.com/frankdejonge
- EventSauce - https://eventsauce.io/
- The Knowledge of Others - https://www.youtube.com/watch?v=RdFGGdL5kH8 -
In this episode of the Cloud Native Compass podcast, the guests discuss WebAssembly and server-side WebAssembly. The guests include Kevin Hoffman, CTO of Cosmonic and creator of the WasmCloud open source project, Connor Hicks, founder and CEO of Suborbital, and Kate Goldenring, senior software engineer at Fermyon.
They each share their experiences and paths to working with WebAssembly, with Kevin having experimented with Asm.js and CloudABI before discovering WebAssembly, Connor being drawn to the performance benefits of WebAssembly for a job processing framework, and Kate coming from a Kubernetes background and working on a project called Krustlet, which was an attempt to run WebAssembly instead of containers on Kubernetes.
The discussion also covers the WebAssembly Systems Interface (WASI), which was originally created to give WebAssembly modules access to low-level external services like file descriptors and standard out, but is evolving towards a component model where external services are high-level abstractions like web clients, web servers, and key value stores. WASI is defined by WIT interfaces that give access to host resources or other resources that the host can control.
Overall, the episode provides insights into the growing use of WebAssembly and server-side WebAssembly in cloud-based applications and the potential benefits they offer.
People
Kate Goldenring - Guest Connor Hicks - Guest Kevin Hoffman - Guest David Flanagan - Host
Creators & Guests
Chapters
(00:00) - Introductions(01:42) - The Path to WebAssembly(06:35) - What is WASI & Component Model(17:20) - Why WebAssembly?(22:35) - When WebAssembly?(34:00) - WebAssembly Adoption(41:40) - Containers with WebAssembly(47:15) - Shameless Plugs
Links
- https://twitter.com/KateGoldenring
- https://fermyon.com
- https://twitter.com/cohix
- https://suborbital.dev
- https://cosmonic.com -
In the first episode of the Cloud Native Compass podcast, the hosts discuss a range of topics related to cloud, containers, and Kubernetes. The conversation begins with a discussion on the cost of cloud adoption, which is becoming increasingly challenging for many organizations. The guests share their experiences and offer advice on how to navigate this tricky situation.
The guests also discuss their backgrounds and experiences in the tech industry. Eli, a developer advocate at Scaleway, shares their experience in data engineering and programming. Abdel, a cloud developer advocate at Google, talks about his experience in datacenters and his focus on Kubernetes and service mesh. Neil, CEO and co-founder of Portainer, describes himself as an engineer and shares his experience in building, breaking, and fixing things. He also emphasises his love for making things easy and his hatred for complexity.
Overall, the podcast provides valuable insights into the cloud native landscape and offers practical advice for organisations looking to adopt cloud technologies. The guests' diverse backgrounds and experiences bring a unique perspective to the conversation, making it an informative and engaging listen for anyone interested in cloud computing.
People
Neil Cresswell - Guest Eli Holderness - Guest Abdellfetah SGHIOUAR - Guest David Flanagan - Host
Creators & Guests
Chapters
(00:00) - Introductions(03:32) - Cost of Cloud(15:35) - Alternative Cloud Providers(26:30) - Do we trust Docker?(37:00) - WebAssembly: Edge, Serverless, More?(49:14) - Internal Developer Platforms (IDPs)(01:03:25) - Shameless Plugs
