Afleveringen
-
The use of large language models (LLMs) has become widespread, but there are significant security risks associated with them. LLMs with millions or billions of parameters are complex and challenging to fully scrutinize, making them susceptible to exploitation by attackers who can find loopholes or vulnerabilities. On an episode of The New Stack Makers, Chris Pirillo, Tech Evangelist and Lance Seidman, Backend Engineer at Atomic Form discussed these security challenges, emphasizing the need for human oversight to protect AI systems.
One example highlighted was malicious AI models on Hugging Face, which exploited the Python pickle module to execute arbitrary commands on users' machines. To mitigate such risks, Hugging Face implemented security scanners to check every file for security threats. However, human vigilance remains crucial in identifying and addressing potential exploits.
Seidman also stressed the importance of technical safeguards and a culture of security awareness within the AI community. Developers should prioritize security throughout the development life cycle to stay ahead of evolving threats. Overall, the message is clear: while AI offers remarkable capabilities, it requires careful management and oversight to prevent misuse and protect against security breaches.
Learn more from The New Stack about AI and security:
Artificial Intelligence: Stopping the Big Unknown in Application, Data Security
Cyberattacks, AI and Multicloud Hit Cybersecurity in 2023
Will Generative AI Kill DevSecOps?
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
The Kubernetes community primarily focuses on improving the development and operations experience for applications and infrastructure, emphasizing DevOps and developer-centric approaches. In contrast, the data science community historically moved at a slower pace. However, with the emergence of the AI engineer persona, the pace of advancement in data science has accelerated significantly.
Alex Williams, founder and publisher of The New Stack co-hosted a discussion with Sanjeev Mohan, an independent analyst, which highlighted the challenges faced by data-related tasks on Kubernetes due to the stateful nature of data. Unlike applications, restarting a database node after a failure may lead to inconsistent states and data loss. This discrepancy in pace and needs between developers and data scientists led to Kubernetes and the Cloud Native Computing Foundation initially overlooking data science.
Nevertheless, Mohan noted that the pace of data engineers has increased as they explore new AI applications and workloads. Kubernetes now plays a crucial role in supporting these advancements by helping manage resources efficiently, especially considering the high cost of training large language models (LLMs) and using GPUs for AI workloads. Mohan also discussed the evolving landscape of AI frameworks and the importance of aligning business use cases with AI strategies. Learn more from The New Stack about data development and DevOps: AI Will Drive Streaming Data Use â But Not Yet, Report Says https://thenewstack.io/ai-will-drive-streaming-data-adoption-says-redpanda-survey/ The Paradigm Shift from Model-Centric to Data-Centric AI https://thenewstack.io/the-paradigm-shift-from-model-centric-to-data-centric-ai/ AI Development Needs to Focus More on Data, Less on Models https://thenewstack.io/ai-development-needs-to-focus-more-on-data-less-on-models/
Learn more from The New Stack about data development and DevOps:
AI Will Drive Streaming Data Use - But Not Yet, Report Says
The Paradigm Shift from Model-Centric to Data-Centric AI
AI Development Needs to Focus More on Data, Less on Models
Join our community of newsletter subscribers to stay on top of the news and at the top of your game. https://thenewstack.io/newsletter/
-
Zijn er afleveringen die ontbreken?
-
LLM observability focuses on maximizing the utility of larger language models (LLMs) by monitoring key metrics and signals. Alex Williams, Founder and Publisher for The New Stack, and Janikiram MSV, Principal of Janikiram & Associates and an analyst and writer for The New Stack, discusses the emergence of the LLM stack, which encompasses various components like LLMs, vector databases, embedding models, retrieval systems, read anchor models, and more. The objective of LLM observability is to ensure that users can extract desired outcomes effectively from this complex ecosystem.
Similar to infrastructure observability in DevOps and SRE practices, LLM observability aims to provide insights into the LLM stack's performance. This includes monitoring metrics specific to LLMs, such as GPU/CPU usage, storage, model serving, change agents in applications, hallucinations, span traces, relevance, retrieval models, latency, monitoring, and user feedback. MSV emphasizes the importance of monitoring resource usage, model catalog synchronization with external providers like Hugging Face, vector database availability, and the inference engine's functionality.
He also mentions peer companies in the LLM observability space like Datadog, New Relic, Signoz, Dynatrace, LangChain (LangSmith), Arize.ai (Phoenix), and Truera, hinting at a deeper exploration in a future episode of The New Stack Makers.
Learn more from The New Stack about LLM and observability
Observability in 2024: More OpenTelemetry, Less Confusion
How AI Can Supercharge Observability
Next-Gen Observability: Monitoring and Analytics in Platform Engineering
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
In a conversation on The New Stack Makers, co-hosted by Alex Williams, TNS founder and publisher, and Charles Humble, an industry expert who served as a software engineer, architect and CTO and now podcaster, author and consultant at Conissaunce Ltd., discussed why software developers and engineers should care about their impact on climate change. Humble emphasized that building software sustainably starts with better operations, leading to cost savings and improved security. He cited past successes in combating environmental issues like acid rain and the ozone hole through international agreements and emissions reduction strategies.
Despite modest growth since 2010, data centers remain significant electricity consumers, comparable to countries like Brazil. The power-intensive nature of AI models exacerbates these challenges and may lead to scarcity issues. Humble mentioned the Green Software Foundation's Maturity Matrix with goals for carbon-free data centers and longer device lifespans, discussing their validity and the role of regulation in achieving them. Overall, software development's environmental impact, primarily carbon emissions, necessitates proactive measures and industry-wide collaboration.
Learn more from The New Stack about sustainability:
What is GreenOps? Putting a Sustainable Focus on FinOps
Unraveling the Costs of Bad Code in Software Development
Can Reducing Cloud Waste Help Save the Planet?
How to Build Open Source Sustainability
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
This New Stack Makers podcast co-hosted by Alex Williams, TNS founder and publisher, and Adrian Cockcroft, Partner and Analyst at OrionX.net, discussed Nvidia's GH200 Grace Hopper superchip. Industry expert Sunil Mallya, Co-founder and CTO of Flip AI weighed in on how it is revolutionizing the hardware industry for AI workloads by centralizing GPU communication, reducing networking overhead, and creating a more efficient system.
Mallya noted that despite its innovative design, challenges remain in adoption due to interface issues and the need for software to catch up with hardware advancements. However, optimism persists for the future of AI-focused chips, with Nvidia leading the charge in creating large-scale coherent memory systems. Meanwhile, Flip AI, a DevOps large language model, aims to interpret observability data to troubleshoot incidents effectively across various cloud platforms. While discussing the latest chip innovations and challenges in training large language models, the episode sheds light on the evolving landscape of AI hardware and software integration.
Learn more from The New Stack about Nvidia and the future of chip design
Nvidia Wants to Rewrite the Software Development Stack
Nvidia GPU Dominance at a Crossroads
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
This New Stack Makers podcast co-hosted by TNS founder and publisher, Alex Williams and Joan Westenberg, founder and writer of Joanâs Index, discussed Copilot. Westenberg highlighted its integration with Microsoft 365 and its role as a coding assistant, showcasing its potential to streamline various tasks.
However, she also revealed its limitations, particularly in reliability. Despite being designed to assist with tasks across Microsoft 365, Copilot's performance fell short during Westenberg's tests, failing to retrieve necessary information from her email and Microsoft Teams meetings. While Copilot proves useful for coding, providing helpful code snippets, its effectiveness diminishes for more complex projects. Westenberg's demonstrations underscored both the strengths and weaknesses of Copilot, emphasizing the need for improvement, especially in reliability, to fulfill its promise as a versatile work companion.
Learn more from The New Stack about Copilot
Microsoft One-ups Google with Copilot Stack for Developers
Copilot Enterprises Introduces Search and Customized Best Practices
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
This New Stack Makers podcast co-hosted by Adrian Cockroft, analyst at OrionX.net and TNS founder and publisher, Alex Williams discusses the importance of monitoring services utilizing Large Language Models (LLMs) and the emergence of tools like LangChain and LangSmith to address this need. Adrian Cockcroft, formerly of Netflix and now working with The New Stack, highlights the significance of monitoring AI apps using LLMs and the challenges posed by slow and expensive API calls from LLMs. LangChain acts as middleware, connecting LLMs with services, akin to the Java Database Controller. LangChain's monitoring capabilities led to the development of LangSmith, a monitoring tool. Another tool, LangKit by WhyLabs, offers similar functionalities but is less integrated. This reflects the typical evolution of open-source projects into commercial products. LangChain recently secured funding, indicating growing interest in such monitoring solutions. Cockcroft emphasizes the importance of enterprise-level support and tooling for integrating these solutions into commercial environments. This discussion underscores the evolving landscape of monitoring services powered by LLMs and the emergence of specialized tools to address associated challenges.
Learn more from The New Stack about LangChain:
LangChain: The Trendiest Web Framework of 2023, Thanks to AI
How Retool AI Differs from LangChain (Hint: It's Automation)
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
In this New Stack Makers podcast, Martin Parker, a solutions architect for UST, spoke with TNS editor-in-chief, Heather Joslyn and discussed the significance of internal developer platforms (IDPs), emphasizing benefits beyond frontend developers to backend engineers and site reliability engineers (SREs).
Parker highlighted the role of IDPs in automating repetitive tasks, allowing SREs to focus on optimizing application performance. Standardization is key, ensuring observability and monitoring solutions align with best practices and cater to SRE needs. By providing standardized service level indicators (SLIs) and key performance indicators (KPIs), IDPs enable SREs to maintain reliability efficiently. Parker stresses the importance of avoiding siloed solutions by establishing standardized practices and tools for effective monitoring and incident response. Overall, the deployment of IDPs aims to streamline operations, reduce incidents, and enhance organizational value by empowering SREs to concentrate on system maintenance and improvements.
Learn more from The New Stack about UST:
Cloud Cost-Unit Economics- A Modern Profitability Model
Cloud Native Users Struggle to Achieve Benefits, Report Says
John our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
In this New Stack Makers podcast, Ben Wilcock, a senior technical marketing architect for Tanzu, spoke with TNS editor-in-chief, Heather Joslyn and discussed the challenges organizations face when building internal developer platforms, particularly the issue of scope, at KubeCon + CloudNativeCon North America.
He emphasized the difficulty for platform engineering teams to select and integrate various Kubernetes projects amid a plethora of options. Wilcock highlights the complexity of tracking software updates, new features, and dependencies once choices are made. He underscores the advantage of having a standardized approach to software deployment, preventing errors caused by diverse mechanisms.
Tanzu aims to simplify the adoption of platform engineering and internal developer platforms, offering a turnkey approach with the Tanzu Application Platform. This platform is designed to be flexible, malleable, and functional out of the box. Additionally, Tanzu has introduced the Tanzu Developer Portal, providing a focal point for developers to share information and facilitating faster progress in platform engineering without the need to integrate numerous open source projects.
Learn more from The New Stack about Tanzu and internal developer platforms:
VMware Unveils a Pile of New Data Services for Its Cloud VMware
VMware Expands Tanzu into a Full Platform Engineering Environment
VMware Targets the Platform Engineer
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
In this New Stack Makers podcast, Mike Stefaniak, senior product manager at NGINX and Kate Osborn, a software engineer at NGINX discusses challenges associated with network ingress in Kubernetes clusters and introduces the Kubernetes Gateway API as a solution. Stefaniak highlights the issues that arise when multiple teams work on the same ingress, leading to friction and incidents. NGINX has also introduced the NGINX Gateway Fabric, implementing the Kubernetes Gateway API as an alternative to network ingress.
The Kubernetes Gateway API, proposed four years ago and recently made generally available, offers advantages such as extensibility. It allows referencing policies with custom resource definitions for better validation, avoiding the need for annotations. Each resource has an associated role, enabling clean application of role-based access control policies for enhanced security.
While network ingress is prevalent and mature, the Kubernetes Gateway API is expected to find adoption in greenfield projects initially. It has the potential to unite North-South and East-West traffic, offering a role-oriented API for comprehensive control over cluster traffic. The article encourages exploring the Kubernetes Gateway API and engaging with the community to contribute to its development.
Learn more from The New Stack about NGINX and the open source Kubernetes Gateway API:
Kubernetes API Gateway 1.0 Goes Live, as Maintainers Plan for The Future
API Gateway, Ingress Controller or Service Mesh: When to Use What and Why
Ingress Controllers or the Kubernetes Gateway API? Which is Right for You?
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
TNS publisher Alex Williams spoke with Ben Kramer, co-founder and CTO of Monterey.ai Cole Hoffer, Senior Software Engineer at Monterey.ai to discuss how the company utilizes vector search to analyze user voices, feedback, reviews, bug reports, and support tickets from various channels to provide product development recommendations. Monterey.ai connects customer feedback to the development process, bridging customer support and leadership to align with user needs. Figma and Comcast are among the companies using this approach.
In this interview, Kramer discussed the challenges of building Large Language Model (LLM) based products and the importance of diverse skills in AI web companies and how Monterey employs Zilliz for vector search, leveraging Milvus, an open-source vector database.
Kramer highlighted Zilliz's flexibility, underlying Milvus technology, and choice of algorithms for semantic search. The decision to choose Zilliz was influenced by its performance in the company's use case, privacy and security features, and ease of integration into their private network. The cloud-managed solution and Zilliz's ability to meet their needs were crucial factors for Monterey AI, given its small team and preference to avoid managing infrastructure.
Learn more from The New Stack about Zilliz and vector database search:
Improving ChatGPTâs Ability to Understand Ambiguous Prompts
Create a Movie Recommendation Engine with Milvus and Python
Using a Vector Database to Search White House Speeches
Join our community of newsletter subscribers to stay on top of the news and at the top of your game. https://thenewstack.io/newsletter/
-
TNS host Heather Joslyn sits down with Ron Masas to discuss trade-offs when it comes to creating fast, secure applications and APIs. He notes a common issue of neglecting documentation and validation, leading to vulnerabilities. Weak authorization is a recurring problem, with instances where changing an invoice ID could expose another user's data.
Masas, an ethical hacker, highlights the risk posed by "zombie" APIsâapplications that have become disused but remain potential targets. He suggests investigating frameworks, checking default configurations, and maintaining robust logging to enhance security. Collaboration between developers and security teams is crucial, with "security champions" in development teams and nuanced communication about vulnerabilities from security teams being essential elements for robust cybersecurity.
For further details, the podcast discusses case studies involving TikTok and Digital Ocean, Masas's views on AI and development, and anticipated security challenges.
Learn more from The New Stack about Imperva and API security:
What Developers Need to Know about Business Logic Attacks
Why Your APIs Arenât Safe â and What to Do about It
The Limits of Shift-Left: Whatâs Next for Developer Security
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
-
Platform engineering âis the art of designing and binding all of the different tech and tools that you have inside of an organization into a golden path that enables self service for developers and reduces cognitive load,â said Kaspar Von GrĂŒnberg, founder and CEO of Humanitec, in this episode of The New Stack Makers podcast.
This structure is important for individual contributors, GrĂŒnberg said, as well as backend engineers: âif you look at the operation teams, it reduces their burden to do repetitive things. And so platform engineers build and design internal developer platforms, and help and serve users."
This conversation, hosted by Heather Joslyn, TNS features editor, dove into platform engineering: what it is, how it works, the problems it is intended to solve, and how to get started in building a platform engineering operation in your organization. It also debunks some key fallacies around the concept.
Learn more from The New Stack about Platform Engineering and Humanitec:
Platform Engineering Overview, News, and Trends
The Hype Train Is Over. Platform Engineering Is Here to Stay
9 Steps to Platform Engineering Hell
-
Is the end of programming nigh? That's the big question posed in this episode recorded earlier in 2023. It was very popular among listeners, and with the topic being as relevant as ever, we wanted to wrap up the year by highlighting this conversation again.
If you ask Matt Welsh, he'd say yes, the end of programming is upon us. As Richard McManus wrote on The New Stack, Welsh is a former professor of computer science at Harvard who spoke at a virtual meetup of the Chicago Association for Computing Machinery (ACM), explaining his thesis that ChatGPT and GitHub Copilot represent the beginning of the end of programming.
Welsh joined us on The New Stack Makers to discuss his perspectives about the end of programming and answer questions about the future of computer science, distributed computing, and more.
Welsh is now the founder of fixie.ai, a platform they are building to let companies develop applications on top of large language models to extend with different capabilities.
For 40 to 50 years, programming language design has had one goal. Make it easier to write programs, Welsh said in the interview.
Still, programming languages are complex, Welsh said. And no amount of work is going to make it simple.
Learn more from The New Stack about AI and the future of software development:
Top 5 Large Language Models and How to Use Them Effectively
30 Non-Trivial Ways for Developers to Use GPT-4
Developer Tips in AI Prompt Engineering
-
Kubevirt, a relatively new capability within Kubernetes, signifies a shift in the virtualization landscape, allowing operations teams to run KVM virtual machines nested in containers behind the Kubernetes API. This integration means that the Kubernetes API now encompasses the concept of virtual machines, enabling VM-based workloads to operate seamlessly within a cluster behind the API. This development addresses the challenge of transitioning traditional virtualized environments into cloud-native settings, where certain applications may resist containerization or require substantial investments for adaptation.
The emerging era of virtualization simplifies the execution of virtual machines without concerning the underlying infrastructure, presenting various opportunities and use cases. Noteworthy advantages include simplified migration of legacy applications without the need for containerization, thereby reducing associated costs.
Kubevirt 1.1, discussed at KubeCon in Chicago by Red Hat's Vladik Romanovsky and Nvidia's Ryan Hallisey, introduces features like memory hotplug and vCPU hotplug, emphasizing the stability of Kubevirt. The platform's stability now allows for the implementation of features that were previously constrained.
Learn more from The New Stack about Kubevirt and the Cloud Native Computing Foundation:
The Future of VMs on Kubernetes: Building on KubeVirt
A Platform for Kubernetes
Scaling Open Source Community by Getting Closer to Users
-
The Kubernetes landscape is evolving, shifting from the domain of visionaries and early adopters to a more mainstream audience. Tigera, represented by CEO Ratan Tipirneni at KubeCon North America in Chicago, recognizes the changing dynamics and the demand for simplified Kubernetes solutions. Tigera's open-source Calico security platform has been updated with a focus on mainstream users, presenting a cohesive and user-friendly solution. This update encompasses five key capabilities: vulnerability scoring, configuration hardening, runtime security, network security, and observability.
The aim is to provide users with a comprehensive view of their cluster's security through a zero to 100 scoring system, tracked over time. Tigera's recommendation engine suggests actions to enhance overall security based on the risk profile, evaluating factors such as egress traffic controls and workload isolation within dynamic Kubernetes environments. Tigera emphasizes the importance of understanding the actual flow of data across the network, using empirical data and observed behavior to build accurate security measures rather than relying on projections. This approach addresses the evolving needs of customers who seek not just vulnerability scores but insights into runtime behavior for a more robust security profile.
Learn more from The New Stack about Tigera and Cloud Native Security:
Cloud Native Network Security: Whoâs Responsible?
Turbocharging Host Workloads with Calico eBPF and XDP
3 Observability Best Practices for Cloud Native App Security
-
Boeing, with around 6,000 engineers, is emphasizing open source engagement by focusing on three main themes, according to Damani Corbin, who heads Boeing's Open Source office. He joined our host, Alex Williams, for a discussion at KubeCon+CloudNativeCon in Chicago.
The first priority Corbin talks about is simplifying the consumption of open source software for developers. Second, Boeing aims to facilitate developer contributions to open source projects, fostering involvement in communities like the Cloud Native Computing Foundation and the Linux Foundation. The third theme involves identifying opportunities for "inner sourcing" to share internally developed solutions across different groups.
Boeing is actively working to break down barriers and encourage code reuse across the organization, promoting participation in open source initiatives. Corbin highlights the importance of separating business-critical components from those that can be shared with the community, prioritizing security and extending efforts to enhance open source security practices. The organization is consolidating its open source strategy by collaborating with legal and information security teams.
Corbin emphasizes the goal of making open source involvement accessible and attractive, with a phased approach to encourage meaningful contributions and ultimately enabling the compensation of engineers for open source work in the future.
Learn more from The New Stack about Boeing and CNCF open source projects:
How Boeing Uses Cloud Native
How Open Source Has Turned the Tables on Enterprise Software
Scaling Open Source Community by Getting Closer to Users
Mercedes-Benz: 4 Reasons to Sponsor Open Source Projects
-
At KubeCon + CloudNativeCon North America 2022, Amazon Web Services (AWS) revealed plans to mirror Kubernetes assets hosted on Google Cloud, addressing Cloud Native Computing Foundation's (CNCF) egress costs. A year later, the project, led by AWS's Davanum Srinivas, redirects image requests to the nearest cloud provider, reducing egress costs for users.
AWS's Todd Neal and Jonathan Innis discussed this on The New Stack Makers podcast recorded at KubeCon North America 2023. Neal explained the registry's functionality, allowing users to pull images directly from the respective cloud provider, avoiding egress costs.
The discussion also highlighted AWS's recent open source contributions, including beta features in Kubectl, prerelease of Containerd 2.0, and Microsoft's support for Karpenter on Azure. Karpenter, an AWS-developed Kubernetes cluster autoscaler, simplifies node group configuration, dynamically selecting instance types and availability zones based on running pods.
The AWS team encouraged developers to contribute to Kubernetes ecosystem projects and join the sig-node CI subproject to enhance kubelet reliability. The conversation in this episode emphasized the benefits of open development for rapid feedback and community collaboration.
Learn more from The New Stack about AWS and Open Source:
Powertools for AWS Lambda Grows with Help of Volunteers
Amazon Web Services Open Sources a KVM-Based Fuzzing Framework
AWS: Why We Support Sustainable Open Source
-
In the past year, developers have faced both promise and uncertainty, particularly in the realm of generative AI. Heath Newburn, global field CTO for PagerDuty, joins TNS host Heather Joslyn to talk about the impact AI and other topics will have on developers in 2024.
Newburn anticipates a growing emphasis on DevSecOps in response to high-profile cyber incidents, noting a shift in executive attitudes toward security spending. The rise of automation-centric tools like Backstage signals a changing landscape in the link between development and operations tools. Notably, there's a move from focusing on efficiency gains to achieving new outcomes, with organizations seeking innovative products rather than marginal coding speed improvements.
Newburn highlights the importance of experimentation, encouraging organizations to identify areas for trial and error, learning swiftly from failures. The upcoming year is predicted to favor organizations capable of rapid experimentation and information gathering over perfection in code writing.
Listen to the full podcast episode as Newburn further discusses his predictions related to platform engineering, remote work, and the continued impact of generative AI.
Learn more from The New Stack about PagerDuty and trends in software development:
How AI and Automation Can Improve Operational Resiliency
Why Infrastructure as Code Is Vital for Modern DevOps
Operationalizing AI: Accelerating Automation, DataOps, AIOps
-
In this episode of The New Stack Makers, Rob Skillington, co-founder and CTO of Chronosphere, discusses the challenges engineers face in building tools for their organizations. Skillington emphasizes that the "build or buy" decision oversimplifies the issue of tooling and suggests that understanding the abstractions of a project is crucial. Engineers should consider where to build and where to buy, creating solutions that address the entire problem. Skillington advises against short-term thinking, urging innovators to consider the long-term landscape.
Drawing from his experience at Uber, Skillington highlights the importance of knowing the audience and customer base, even when they are colleagues. He shares a lesson learned when building a visualization platform for engineers at Uber, where understanding user adoption as a key performance indicator upfront could have improved the project's outcome.
Skillington also addresses the "not invented here syndrome," noting its prevalence in organizations like Microsoft and its potential impact on tool adoption. He suggests that younger companies, like Uber, may be more inclined to explore external solutions rather than building everything in-house. The conversation provides insights into Skillington's experiences and the considerations involved in developing internal tools and platforms.
Learn more from The New Stack about Software Engineering, Observability, and Chronosphere:
Cloud Native Observability: Fighting Rising Costs, Incidents
A Guide to Measuring Developer Productivity
4 Key Observability Best Practices
- Laat meer zien
