apache2-licensed
A Python package & command-line tool to gather text on the Web.
Trafilatura is a Python package and command-line tool designed to gather text on the Web. It includes discovery, extraction and text processing components. Its main applications are web crawling, downloads, scraping, and extraction of main texts, metadata and comments. It aims at staying handy and modular: no database is required, the output can be converted to various commonly used formats.
Related contents:
Linux Native, API-Aware Networking and Security for Containers. eBPF-based Networking, Observability, Security.
Cilium is a networking, observability, and security solution with an eBPF-based dataplane. It provides a simple flat Layer 3 network with the ability to span multiple clusters in either a native routing or overlay mode. It is L7-protocol aware and can enforce network policies on L3-L7 using an identity based security model that is decoupled from network addressing.
Related contents:
Superset is a modern data exploration and data visualization platform. Superset can replace or augment proprietary business intelligence tools for many teams. Superset integrates well with a variety of data sources.
Fast Open-Source OLAP DBMS.
ClickHouse® is an open-source column-oriented database management system that allows generating analytical data reports in real-time.
Related contents:
- Altinity Kubernetes Operator for ClickHouse @ GitHub.
- ClickHouse on Kubernetes @ Sr. Data Engineer.
- Inside ClickHouse full-text search: fast, native, and columnar @ ClickHouse.
- How we made ClickHouse log queries 99.5% faster with resource fingerprinting @ SigNoz.
- From Millions to Billions @ geocodio.
- The KFC Architecture Blueprint: Kafka, Flink, and ClickHouse @ Big Data Boutique.
- How we give every user SQL access to a shared ClickHouse cluster @ Trigger.dev.
Deploy a Production Ready Kubernetes Cluster
-
Can be deployed on AWS, GCE, Azure, OpenStack, vSphere, Equinix Metal (bare metal), Oracle Cloud Infrastructure (Experimental), or Baremetal
-
Highly available cluster
-
Composable (Choice of the network plugin for instance)
-
Supports most popular Linux distributions
-
Continuous integration tests
simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system. Mountpoint for Amazon S3 is a simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system. With Mountpoint for Amazon S3, your applications can access objects stored in Amazon S3 through file operations like open and read. Mountpoint for Amazon S3 automatically translates these operations into S3 object API calls, giving your applications access to the elastic storage and throughput of Amazon S3 through a file interface.
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
Kopia is a fast and secure open-source backup/restore tool that allows you to create encrypted snapshots of your data and save the snapshots to remote or cloud storage of your choice, to network-attached storage or server, or locally on your machine. Kopia does not 'image' your whole machine. Rather, Kopia allows you to backup/restore any and all files/directories that you deem are important or critical.
Related contents:
A distributed, reliable key-value store for the most critical data of a distributed system.
etcd is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. It gracefully handles leader elections during network partitions and can tolerate machine failure, even in the leader node. Learn more.
Related contents:
DNS and Service Discovery.
CoreDNS is a DNS server. It is written in Go. It can be used in a multitude of environments because of its flexibility.
CoreDNS is a DNS server/forwarder, written in Go, that chains plugins. Each plugin performs a (DNS) function.
CoreDNS can listen for DNS requests coming in over UDP/TCP (go'old DNS), TLS (RFC 7858), also called DoT, DNS over HTTP/2 - DoH - (RFC 8484) and gRPC (not a standard).
Related contents:
Qdrant (read: quadrant ) is a vector similarity search engine and vector database. It provides a production-ready service with a convenient API to store, search, and manage points - vectors with an additional payload. Qdrant is tailored to extended filtering support. It makes it useful for all sorts of neural-network or semantic-based matching, faceted search, and other applications.
Related contents:
Woodpecker is a simple CI engine with great extensibility. Woodpecker is a community fork of the Drone CI system.
TypeScript is JavaScript with syntax for types. TypeScript is a strongly typed programming language that builds on JavaScript, giving you better tooling at any scale.
TypeScript is a language for application-scale JavaScript. TypeScript adds optional types to JavaScript that support tools for large-scale JavaScript applications for any browser, for any host, on any OS. TypeScript compiles to readable, standards-based JavaScript.
Related contents:
- The Concise TypeScript Book.
- Hyper-Typing @ Paolo Scanferla.
- Identify and Fix Code Smells in TypeScript @ Engroso's dev.to.
- Satisfies in TypeScript @ Frontend Masters Blog.
- The Temporal Dead Zone, or why the TypeScript codebase is littered with var statements @ Vincent Rolfs.
- TypeScript’s rise in the AI era: Insights from Lead Architect, Anders Hejlsberg @ GitHub Blog.
- The Inner Workings of JavaScript Source Maps @ Polar Signals.
- Experiment: making TypeScript immutable-by-default @ Evan Hahn.
- TIL:
satisfiesis my favorite TypeScript keyword @ Jerred Shepherd. - Stop writing if statements for your CLI flags @ Hackers' Pub.
- JSDoc is TypeScript @ culi.bear.blog.
- I was wrong about TypeScript: Part 1 @ Amadu's & Don'ts.
- Fixing TypeScript Performance Problems: A Case Study @ viget.
- TypeScript or Tears @ nicolas.vbgh's dev.to.
- Seven Years to TypeScript: Migrating 11,000 Files at Patreon @ Engineering at Patreon.
- Parse, Don't Validate — In a Language That Doesn't Want You To @ Cekrem.GitHub.IO.
Meet Zed – The fast, collaborative code editor.
Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter. It's also open source.
Related contents:
A distributed vulnerability database for Open Source. An open, precise, and distributed approach to producing and consuming vulnerability information for open source.
Related contents:
A truly Open Source MongoDB alternative. FerretDB (previously MangoDB) was founded to become the de-facto open-source substitute to MongoDB. FerretDB is an open-source proxy, converting the MongoDB 6.0+ wire protocol queries to SQL - using PostgreSQL as a database engine.
Related contents:
- Microsoft builds open source document database on PostgreSQL, suggests FerretDB as front end @ The Register.
- Building for Developers—Not Imitators @ MongoDB.
- Oxygen Deprivation: FerretDB with Peter Farkas @ Contributor.fyi.
- FerretDB Was Eating My CPU: Migrating Komodo from SQLite to Postgres @ mauveRANT.
Transform your Messy CI Scripts into Clean Code. Develop Your CI/CD Pipelines as Code and Run Them Anywhere.
Powerful, programmable open source CI/CD engine that runs your pipelines in containers — pre-push on your local machine and/or post-push in CI. Dagger is a tool that lets you replace your software project's artisanal scripts with a modern API and cross-language scripting engine.
Related contents:
Software to automate the management and configuration of infrastructure and applications at scale.
Salt is the world’s fastest, most intelligent and scalable automation engine. Built on python, Salt uses simple and human-readable YAML combined with event-driven automation to deploy and configure complex IT systems. In addition to leveling-up vRealize Automation SaltStack Config, Salt can be found under the hood of products from Juniper, Cisco, Cloudflare, Nutanix, SUSE, and Tieto, to name a few.
Related contents:
Crack your data into pieces. Seamlessly visualize your JSON data instantly into graphs.
✨ Innovative and open-source visualization application that transforms various data formats, such as JSON, YAML, XML, CSV and more, into interactive graphs.
Open Computer Vision. Open source machine learning library for computer vision.
Related contents:
Open Cyber Threat Intelligence Platform.
OpenCTI is an open source platform allowing organizations to manage their cyber threat intelligence knowledge and observables. It has been created in order to structure, store, organize and visualize technical and non-technical information about cyber threats.
Related contents:
Detect security threats in real time
Falco is a cloud-native security tool designed for Linux systems. It employs custom rules on kernel events, which are enriched with container and Kubernetes metadata, to provide real-time alerts. Falco helps you gain visibility into abnormal behavior, potential security threats, and compliance violations, contributing to comprehensive runtime security.
Related contents:
- k8s-custom-detections @ GitHub.
- Réagir à temps aux menaces dans Kubernetes avec Falco (Rachid Zarouali, Thomas Labarussias) @ Voxxed Days Luxembourg's YouTube :fr:.
- Extending Falco for Bitcoin @ sysdig.
- Falcoctl: Artifact Management for Falco @ Cloud Native Computing Foundation.
- How Falco and Stratoshark close the gap between open source runtime detection and deep forensic analysis @ sysdig.
- Blueprinting Security in CI/CD: Building Trust Through Open Source @ CD Foundation.
- Episode #540 consacré à Falco @ NoLimitSecu :fr:.
Enterprise-grade open source web application firewall library. Coraza is an open source, high performance, Web Application Firewall ready to protect your beloved applications.
Podman is a daemonless container engine for developing, managing, and running OCI Containers on your Linux System. Containers can either be run as root or in rootless mode. Simply put: alias docker=podman.
Related contents:
- Replacing Kubernetes with systemd @ Yaakov's Blog.
- Auto-updating podman containers with systemd @ fedora Magazine.
- Connecter un conteneur Podman à Open vSwitch @ Nicolas Massé :fr:.
- Why I Ditched Docker for Podman (And You Should Too) @ Codesmash.
- Automatic container updates with Podman quadlets @ Major Hayden.
- podman : Utiliser podman comme docker avec les mêmes commandes @ Linuxtricks.fr :fr:.
- The Docker Alternative Most People DON'T KNOW @ CodeHead's YouTube.
- Grand saut dans le déploiement sur site d'un serveur LLM à moindres privilèges @ Synacktiv :fr:.
🐶 Kubernetes CLI To Manage Your Clusters In Style!
K9s is a terminal based UI to interact with your Kubernetes clusters. The aim of this project is to make it easier to navigate, observe and manage your deployed applications in the wild. K9s continually watches Kubernetes for changes and offers subsequent commands to interact with your observed resources.
The Distributed Application Runtime (Dapr) provides APIs that simplify microservice connectivity. Whether your communication pattern is service to service invocation or pub/sub messaging, Dapr helps you write resilient and secured microservices.
Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.
Related contents:
A container-based solution for automating Docker container base image updates.
With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.
Containerrr's Watchtower fork.
Securely connect anything with WireGuard® and manage all your networks from a single place. Drago is a flexible configuration manager for WireGuard designed to make it simple to configure secure network overlays spanning heterogeneous nodes distributed across different clouds and physical locations.
Related contents:
Our mission is to be the trusted cloud native repository for Kubernetes. Harbor is an open source registry that secures artifacts with policies and role-based access control, ensures images are scanned and free from vulnerabilities, and signs images as trusted. Harbor, a CNCF Graduated project, delivers compliance, performance, and interoperability to help you consistently and securely manage artifacts across cloud native compute platforms like Kubernetes and Docker.
Related contents:
Airflow is a platform created by the community to programmatically author, schedule and monitor workflows.
Related contents:
- Apache Airflow Configuration and Tuning @ DZone.
- Exploring Apache Airflow for Batch Processing Scenario @ DZone.
- What Is Apache Airflow @ Seattle Data Guy.
- Alternatives to Talend – How To Migrate Away From Talend For Your Data Pipelines @ Seattle Data Guy.
- Improving workflow orchestration with Apache Airflow 3.1 in Cloud Composer @ Google Cloud Blog.
- How You can Automate PostgreSQL Backups to S3 Using Apache Airflow! @ Kube Blogs.
- Générer un Excel volumineux avec Airflow sans dépassement de mémoire @ .LOUD :fr:.
Database management in a single PHP file.
Adminer (formerly phpMinAdmin) is a full-featured database management tool written in PHP. Conversely to phpMyAdmin, it consist of a single file ready to deploy to the target server. Adminer is available for MySQL, MariaDB, PostgreSQL, SQLite, MS SQL, Oracle, Elasticsearch, MongoDB and others via plugin.
🔎 Open source distributed and RESTful search engine. OpenSearch makes it easy to ingest, search, visualize, and analyze your data.
OpenSearch is a community-driven, open source fork of Elasticsearch and Kibana following the license change in early 2021. We're looking to sustain (and evolve!) a search and analytics suite for the multitude of businesses who are dependent on the rights granted by the original, Apache v2.0 License.
Related contents:
The world's most widely used web app scanner. Free and open source. Actively maintained by a dedicated international team of volunteers.
Related contents:
A data templating language for app and tool developers.
-
Generate config data
-
Side-effect free
-
Organize, simplify, unify
-
Manage sprawling config
Related contents:
Messaging that just works. RabbitMQ is the most widely deployed open source message broker.
Related contents:
Checkov scans cloud infrastructure configurations to find misconfigurations before they're deployed.
Checkov uses a common command line interface to manage and analyze infrastructure as code (IaC) scan results across platforms such as Terraform, CloudFormation, Kubernetes, Helm, ARM Templates and Serverless framework.
Related contents:
Maru is a new kind of computing experience. It gives you a single, context-aware device that makes personal computing really simple. And guess what? That device is your smartphone.
TiDB is a distributed NewSQL database compatible with MySQL protocol TiDB (The pronunciation is: /'taɪdiːbi:/ tai-D-B, etymology: titanium) is a distributed SQL database. Inspired by the design of Google F1, TiDB supports the best features of both traditional RDBMS and NoSQL.
Related contents:
the JavaScript Database that Syncs! PouchDB is an open-source JavaScript database inspired by Apache CouchDB that is designed to run well within the browser.
PouchDB was created to help web developers build applications that work as well offline as they do online.
Related contents:
frp is a fast reverse proxy that allows you to expose a local server located behind a NAT or firewall to the Internet. It currently supports TCP and UDP, as well as HTTP and HTTPS protocols, enabling requests to be forwarded to internal services via domain name.
The HTTP/2 Web Server with Fully Managed TLS (automatic HTTPS).
Caddy 2 is a powerful, enterprise-ready, open source web server with automatic HTTPS written in Go
Kinto is a lightweight JSON storage service with synchronisation and sharing abilities. It is meant to be easy to use and easy to self-host.
Kinto is used at Mozilla and released under the Apache v2 licence.