Oh and btw: this account will be on hiatus until further notice. #takingarest

I formally submit my application for the role of CEO of HTMX @htmx_org @gitpush_gitpaid

Happy new year 🎊 every person

Big pricing change to a valuable GCP security feature for least-privilege IAM. Requiring the org-level SCC Premium bundle instead of a pay-as-you-go per-service price will be prohibitively expensive and detrimental to security for many customers in the long run. @philvenables

As of next year, #google will require premium subscriptions for things like IAM Recommender, Role Insights, etc. This is the OPPOSITE of their mantra calling for shared fate instead of the shared responsibility model. A massive loss in credibility.

I just uploaded a 90 minute tutorial, which is designed to be the one place I point coders at when they ask "hey, tell me everything I need to know about LLMs!" It starts at the basics: the 3-step pre-training / fine-tuning / classifier ULMFiT approach used in all modern LLMs.

This paper blew my mind 🤯 Introducing LongLoRA! An ultra efficient fine-tuning method designed to extend the context sizes of pre-trained large language models (LLMs) without a huge computation cost. Typically, training LLMs with longer context sizes consumes a lot of time and requires strong GPU resources. For example, extending the context length from 2048 to 8192 increases computational costs 16 times, particularly in self-attention layers. 🖥️ What sets LongLoRA apart is its two-pronged approach to speeding up the context extension of LLMs. First, it uses sparse local attention instead of dense global attention during the fine-tuning phase, which is a more efficient way to handle this task. This change, known as shift-short attention, significantly saves computational effort while maintaining similar performance levels compared to traditional attention mechanisms. Plus, it’s simple to implement, requiring just two lines of code during training, and it's optional during inference. 💡 Second, LongLoRA revisits the idea of a more parameter-efficient fine-tuning process for context expansion. The effectiveness of LoRA is enhanced when combined with trainable embedding and normalization, showing solid results. In practical terms, LongLoRA showed strong performance on various tasks using LLaMA2 models ranging from 7B/13B to 70B. Notably, it extended LLaMA2 7B from 4k context to 100k, and LLaMA2 70B to 32k on a single 8x A100 machine, all while keeping the original model architectures intact. 📈 Moreover, LongLoRA plays well and compatible with existing techniques like FlashAttention-2. To make it more user-friendly, a dataset called LongQA was created for supervised fine-tuning, containing over 3k long context question-answer pairs. LongLoRA is an important step toward making model expansion more computationally efficient. Paper - arxiv.org/abs/2309.12307 GitHub - github.com/dvlab-research/Lo…

Issue 154: mailchi.mp/deararchitects/15… This week: 1. How to bridge silos inside your organization 2. #devops best practices 3. How @stripe is designed for #BlackFriday traffic 4. New #serverless book from @sheenbrisals and @level_out 5. @OReillyMedia #architectures #books #bundle

Ever wonder what LLMs are for, aside from coding? Here’s one theory:

Here are 8 key considerations for building *production-grade* LLM apps over your data (RAG) 💡 (see 🧵): 1️⃣ Chunks used for retrieval shouldn’t necessarily be the same as chunks used for LLM synthesis (@maxrumpf) 2️⃣ Embeddings should live in a different latent space than what you get from raw text, which can contain filler that biases the embeddings. Consider passing a transformed text representation or finetuning. (@maxrumpf) 3️⃣ If retrieval isn’t returning the right context, you may need to dynamically load/update the data itself (@maxrumpf, @bobvanluijt) 4️⃣ Design your pipeline for scalability. Latency times in prototyping do not translate to production workloads. Start with easy-to-use, high latency modules but aggressively try to shrink latency component by component (@bobvanluijt) 5️⃣ Store data in a hierarchical fashion: summaries of documents and chunks for each document (@bobvanluijt) 6️⃣ Robust data pipelines in prod don't matter as much if you only load the data once. But they are especially important when the source data is constantly changing (@tuanacelik, @maxrumpf) 7️⃣ RAG isn’t just question-answering: for summaries, you may need ALL chunks, for question-answering, you may need specific chunks. Your chunk sizes may differ depending on the use case (@tuanacelik) 8️⃣ Embedding-based retrieval does not work well for entity lookup. Hybrid search combines benefits of keyword lookup with additional context (@tuanacelik) All of these insights and more can be found in our fantastic “Make RAG Production-Ready” webinar, now available for viewing on YouTube! 🔥👇 HUGE shoutout to @tuanacelik, @maxrumpf, and @bobvanluijt for a fantastic discussion. It’s clear that we’ve barely scratched the surface - we’ll definitely be doing a part 2! piped.video/Zj5RCweUHIk

For anynone with a Canon DSLR, that just won't power on after prolonged storage. This fixed it for me: piped.video/watch?v=bkjUJYHD… tl;dr: Remove battery, hold down the shutter button for about 30-60sec, reinsert battery, power on

It's always DNS... #grmpf

Abendlektüre MaRisk und BAIT... Man gönnt sich ja sonst nichts

Smack in the middle of puberty #MyTwitterAnniversary

RT @katjaberlin: wenn ihnen diese harmlosen aktionen der letzten generation schon zu radikal sind, dann habe ich ganz schlechte nachrichten…

Very true that ChatGPT reflects the basic position of the internet back to us. And what that tells us that it is fluent is basic bro-speak. Not logic or reason and it can’t think. It’s basically like a brilliant all-knowing 14 yo boy playing an unending video game

Mal eine Frage an die Gewerbetreibenden unter euch: Wen würdet ihr denn für ein Geschäftskonto empfehlen? Und vielleicht sogar wichtiger: wen nicht?

Excuse me, wir haben 2023! Könnt ich bitte endlich mal einen funktionierenden Realtek Treiber für Linux haben? :/