[{"data":1,"prerenderedAt":356},["ShallowReactive",2],{"tag-labels":3,"site-header":4,"article-rag-pisah-jalur-query-dan-ingestion-id":72},{},{"featuredPost":5,"navigations":29},{"id":6,"title":7,"title_en":8,"slug":9,"excerpt":10,"excerpt_en":11,"has_translation":12,"published_at":13,"reading_time":14,"hidden_from_listing":15,"tags":16,"thumbnail_image":21,"og_image":22,"series":23,"authors":28},110,"Ketika Pilih Framework Cuma Biar Kelihatan Beda","When You Pick a Framework Just to Look Different","ketika-pilih-framework-cuma-biar-kelihatan-beda","Terakhir kali kamu milih framework atau library baru, itu karena masalahmu butuh alat itu, atau karena alatnya kedengaran keren dan belum banyak yang pakai? Pertanyaan ini nggak enak dijawab. Soalnya keputusan teknologi sering diambil dengan alasan yang nggak pernah ditulis di dokumen teknis mana pun.","The last time you chose a new framework or library, was it because your problem needed that tool, or because it sounded cool and few people around you were using it? That question is uncomfortable. Technology decisions are often made for reasons that never appear in any technical document.",true,"2026-10-06T15:25:21.000Z",5,false,[17,18,19,20],"karier","framework","opini","pengalaman",null,"\u002Fog\u002Fketika-pilih-framework-cuma-biar-kelihatan-beda.png",{"id":24,"title":25,"title_en":26,"slug":27},9,"Artikel","Articles","all",[],[30,46,67],{"collection":31,"item":32},"series",{"__typename":31,"title":25,"title_en":26,"slug":27,"children_series":33},[34,38,42],{"title":35,"title_en":36,"slug":37},"Kecerdasan Artifisial","Artificial Intelligence","kecerdasan-artifisial",{"title":39,"title_en":40,"slug":41},"Konsep Dasar Pemrograman","Programming Fundamentals","konsep-dasar-pemrograman",{"title":43,"title_en":44,"slug":45},"Algoritma dan Struktur Data","Algorithms and Data Structures","algoritma-struktur-data",{"collection":31,"item":47},{"__typename":31,"title":48,"title_en":48,"slug":49,"children_series":50},"Tutorial","tutorial",[51,55,59,63],{"title":52,"title_en":53,"slug":54},"Belajar Python","Learning Python","belajar-python",{"title":56,"title_en":57,"slug":58},"Belajar Golang","Learning Go","belajar-golang",{"title":60,"title_en":61,"slug":62},"Belajar Java","Learning Java","belajar-java",{"title":64,"title_en":65,"slug":66},"Belajar Pascal","Learning Pascal","belajar-pascal",{"collection":31,"item":68},{"__typename":31,"title":69,"title_en":69,"slug":70,"children_series":71},"Lab","lab",[],{"article":73,"renderedBody":265,"renderedTakeaway":266,"latestArticles":267,"toc":334},{"id":74,"title":75,"title_en":76,"slug":77,"excerpt":78,"excerpt_en":79,"has_translation":12,"published_at":80,"reading_time":14,"hidden_from_listing":15,"tags":81,"thumbnail_image":21,"og_image":88,"series":89,"authors":257,"body_locale":260,"available_locales":261,"body":263,"takeway":264,"prerequisite":21},129,"Biar RAG nggak kena 504: pisahin jalur query dan ingestion","So Your RAG Stops Hitting 504: Split the Query Path From Ingestion","rag-pisah-jalur-query-dan-ingestion","Satu service RAG yang merangkap parsing dan menjawab kueri bakal tumbang pas ada yang upload PDF 200 halaman. Obatnya dua jalur dan satu queue.","One RAG service that both parses uploads and answers queries will fall over the moment someone uploads a 200-page PDF. The cure is two paths and one queue.","2026-10-22T01:30:00.000Z",[82,83,84,85,86,87],"rag","cloud-run","cloud-tasks","gcp","serverless","arsitektur","\u002Fog\u002Frag-pisah-jalur-query-dan-ingestion.png",{"id":90,"title":35,"title_en":36,"slug":37,"description":21,"description_en":91,"articles":92,"children_series":249,"parent_series":250},8,"Notes on AI that go past the demo: how the models work, where they break, and what changes once you put them in production.",[93,110,128,143,157,171,188,205,225,244],{"id":94,"title":95,"title_en":96,"slug":97,"excerpt":98,"excerpt_en":99,"has_translation":12,"published_at":100,"reading_time":101,"hidden_from_listing":15,"tags":102,"thumbnail_image":21,"og_image":107,"series":108,"authors":109},1,"Kenapa AI Suka 'Halu'? Kenalan sama JEPA, Otak AI yang Punya Akal Sehat","Why Does AI Keep Hallucinating? Meet JEPA, the AI Brain With Common Sense","kenapa-ai-suka-halu-kenalan-sama-jepa-otak-ai-yang-punya-akal-sehat","AI generatif sering 'halu' karena cuma jago nebak pola kata. JEPA coba ngerombak itu dengan ngajarin AI cara kerja dunia nyata kayak otak manusia.","Generative AI hallucinates because all it does is guess word patterns. JEPA tries to change that by teaching machines how the real world works, the way a human brain does.","2026-06-02T08:09:05.595Z",4,[103,104,105,106],"jepa","artificial-intelligence","yann-lecun","agi","\u002Fog\u002Fkenapa-ai-suka-halu-kenalan-sama-jepa-otak-ai-yang-punya-akal-sehat.png",{"id":90,"title":35,"title_en":36,"slug":37},[],{"id":111,"title":112,"title_en":113,"slug":114,"excerpt":115,"excerpt_en":116,"has_translation":12,"published_at":117,"reading_time":101,"hidden_from_listing":15,"tags":118,"thumbnail_image":21,"og_image":120,"series":121,"authors":122},2,"LLM Bakal Mentok, Terus Gimana Dong?","LLMs Are Going to Hit a Wall. What Then?","llm-bakal-mentok-terus-gimana-dong","LLM kayak ChatGPT itu jago ngerangkai kata, tapi nggak beneran paham realitas. Yann LeCun bilang arsitektur ini bakal mentok dan butuh solusi baru.","LLMs like ChatGPT are great at stringing words together, but they never really grasp reality. Yann LeCun says this architecture will hit a wall and needs a new answer.","2026-06-03T08:55:33.003Z",[104,119,105,103],"llm","\u002Fog\u002Fllm-bakal-mentok-terus-gimana-dong.png",{"id":90,"title":35,"title_en":36,"slug":37},[123],{"authors_id":124},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},"Daffa Izzuddin","izzudd","Writer",{"id":129,"title":130,"title_en":131,"slug":132,"excerpt":133,"excerpt_en":134,"has_translation":12,"published_at":135,"reading_time":101,"hidden_from_listing":15,"tags":136,"thumbnail_image":21,"og_image":138,"series":139,"authors":140},3,"Mengenal JEPA: Saat AI Mulai Belajar Punya \"Akal Sehat\"","Meet JEPA: Where AI Starts Learning Common Sense","kenapa-ai-suka-halu-dan-gimana-jepa-beresin-masalah-ini","AI sering 'halu' karena cuma nebak probabilitas data tanpa paham realita. Arsitektur JEPA mencoba mengubah itu dengan membangun model dunia di kepalanya.","AI hallucinates because it only guesses data probabilities without understanding reality. The JEPA architecture tries to change that by building a world model in its head.","2026-06-03T09:32:50.446Z",[104,103,137,105],"world-models","\u002Fog\u002Fkenapa-ai-suka-halu-dan-gimana-jepa-beresin-masalah-ini.png",{"id":90,"title":35,"title_en":36,"slug":37},[141],{"authors_id":142},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":101,"title":144,"title_en":145,"slug":146,"excerpt":147,"excerpt_en":148,"has_translation":12,"published_at":149,"reading_time":101,"hidden_from_listing":15,"tags":150,"thumbnail_image":21,"og_image":152,"series":153,"authors":154},"Bukan Sekadar Generative AI, JEPA Coba Kejar Pemahaman Konsep ala Manusia","Beyond Generative AI: JEPA Chases Human-Style Conceptual Understanding","kenapa-ai-masih-gagal-paham-fisika-dasar-mengenal-arsitektur-jepa","AI biasanya cuma jago nebak piksel, tapi JEPA coba belajar lewat konsep abstrak di ruang laten. Ternyata, cara ini bisa sampai dipakai buat baca gelombang otak.","AI usually only guesses pixels, while JEPA tries to learn from abstract concepts in latent space. That approach turns out to reach as far as reading brain waves.","2026-06-03T09:34:14.840Z",[103,105,151,106],"machine-learning","\u002Fog\u002Fkenapa-ai-masih-gagal-paham-fisika-dasar-mengenal-arsitektur-jepa.png",{"id":90,"title":35,"title_en":36,"slug":37},[155],{"authors_id":156},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":14,"title":158,"title_en":159,"slug":160,"excerpt":161,"excerpt_en":162,"has_translation":12,"published_at":163,"reading_time":101,"hidden_from_listing":15,"tags":164,"thumbnail_image":21,"og_image":166,"series":167,"authors":168},"Di Balik Dapur JEPA: Rahasia EMA, VICReg, dan Efisiensi Komputasi","Inside JEPA's Kitchen: The Secrets of EMA, VICReg, and Compute Efficiency","di-balik-dapur-jepa-rahasia-ema-vicreg-dan-efisiensi-komputasi","Bosen sama AI yang cuma jago nebak kata atau ngafalin pixel? Kenalan sama JEPA, arsitektur revolusioner yang ngajarin mesin punya \"akal sehat\" secara super efisien. Yuk intip jeroan teknisnya!","Bored of AI that only guesses words or memorizes pixels? Meet JEPA, an architecture that teaches machines common sense very cheaply. Here is a look at its technical insides.","2026-06-07T16:14:13.723Z",[103,165],"artificial-intelligent","\u002Fog\u002Fdi-balik-dapur-jepa-rahasia-ema-vicreg-dan-efisiensi-komputasi.png",{"id":90,"title":35,"title_en":36,"slug":37},[169],{"authors_id":170},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":172,"title":173,"title_en":174,"slug":175,"excerpt":176,"excerpt_en":177,"has_translation":12,"published_at":178,"reading_time":90,"hidden_from_listing":15,"tags":179,"thumbnail_image":21,"og_image":185,"series":186,"authors":187},125,"MCP dibongkar ulang: kabar baik buat kamu yang capek nulis konektor","MCP, Rebuilt: Good News If You're Tired of Writing Connectors","mcp-dibongkar-ulang-kabar-baik-buat-kamu","Kamu punya beberapa API internal dan satu agent AI, tapi kodenya harus ditulis ulang tiap kali modelnya diganti. MCP datang buat mengakhiri itu, dan tahun ini protokolnya dibongkar ulang supaya server-nya bisa jalan seperti web server biasa. Yuk lihat apa yang berubah. 🔧","You have a few internal APIs and one AI agent, but the code has to be rewritten every time the model changes. MCP ends that, and this year the protocol was rebuilt so servers can run like ordinary web servers. Here's what changed. 🔧","2026-10-16T05:00:00.000Z",[180,181,182,119,183,184],"mcp","model-context-protocol","ai-agent","backend","python","\u002Fog\u002Fmcp-dibongkar-ulang-kabar-baik-buat-kamu.png",{"id":90,"title":35,"title_en":36,"slug":37},[],{"id":189,"title":190,"title_en":191,"slug":192,"excerpt":193,"excerpt_en":194,"has_translation":12,"published_at":195,"reading_time":196,"hidden_from_listing":15,"tags":197,"thumbnail_image":21,"og_image":200,"series":201,"authors":202},126,"Memori AI Agent: dari Vector Store ke Temporal Graph","AI Agent Memory: From Vector Store to Temporal Graph","memori-ai-agent-vector-store-ke-temporal-graph","Dua fakta yang saling bertentangan bikin agent menjawab ngawur. Ini bedanya vector store dan temporal knowledge graph dalam menjaga state jangka panjang.","Two contradictory facts make an agent answer nonsense. Here is how a vector store and a temporal knowledge graph differ at keeping long-term state.","2026-10-18T13:13:29.458Z",7,[198,119,183,199],"ai","system-design","\u002Fog\u002Fmemori-ai-agent-vector-store-ke-temporal-graph.png",{"id":90,"title":35,"title_en":36,"slug":37},[203],{"authors_id":204},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":206,"title":207,"title_en":208,"slug":209,"excerpt":210,"excerpt_en":211,"has_translation":12,"published_at":212,"reading_time":213,"hidden_from_listing":15,"tags":214,"thumbnail_image":21,"og_image":220,"series":221,"authors":222},127,"Kenapa RAG yang cuma andalkan cosine similarity sering nyasar","Why RAG That Leans on Cosine Similarity Alone Keeps Missing","kenapa-rag-cuma-cosine-similarity-sering-nyasar","RAG yang cuma andalkan embedding balikin chunk yang mirip maknanya, bukan yang kamu butuh. Ini penyebabnya, dan dua lapis penambalnya.","RAG that leans only on embeddings returns chunks that look close in meaning, not the ones you need. Here is why, and the two layers that patch it.","2026-10-19T02:04:06.718Z",6,[82,215,216,217,218,219],"vector-search","bm25","hybrid-search","retrieval","embedding","\u002Fog\u002Fkenapa-rag-cuma-cosine-similarity-sering-nyasar.png",{"id":90,"title":35,"title_en":36,"slug":37},[223],{"authors_id":224},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":226,"title":227,"title_en":228,"slug":229,"excerpt":230,"excerpt_en":231,"has_translation":12,"published_at":232,"reading_time":213,"hidden_from_listing":15,"tags":233,"thumbnail_image":21,"og_image":239,"series":240,"authors":241},128,"Linear atau graf: cara mengatur banyak agent tanpa loop abadi","Linear or Graph: How to Run Many Agents Without an Eternal Loop","multi-agen-linear-atau-grafik","Satu agent dengan dua puluh tool gampang kacau. Begitu beberapa agent saling revisi, kamu butuh graf yang mendukung cycle dan angka yang membuatnya berhenti.","One agent with twenty tools gets messy fast. Once several agents revise each other, you need a graph that allows cycles and a number that makes it stop.","2026-10-21T16:05:00.000Z",[234,235,236,237,238,119],"multi-agent","langgraph","autogen","orkestrasi","checkpoint","\u002Fog\u002Fmulti-agen-linear-atau-grafik.png",{"id":90,"title":35,"title_en":36,"slug":37},[242],{"authors_id":243},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":74,"title":75,"title_en":76,"slug":77,"excerpt":78,"excerpt_en":79,"has_translation":12,"published_at":80,"reading_time":14,"hidden_from_listing":15,"tags":81,"thumbnail_image":21,"og_image":88,"series":245,"authors":246},{"id":90,"title":35,"title_en":36,"slug":37},[247],{"authors_id":248},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},[],{"id":24,"title":25,"title_en":26,"slug":27,"parent_series":21,"children_series":251},[252,253,255],{"id":90,"title":35,"title_en":36,"slug":37,"cover_image":21},{"id":254,"title":39,"title_en":40,"slug":41,"cover_image":21},12,{"id":256,"title":43,"title_en":44,"slug":45,"cover_image":21},15,[258],{"authors_id":259},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},"id",[262,260],"en","\nBikin RAG di masa hackathon itu terasa instan. Satu container FastAPI, terima file PDF, ekstrak teks, bikin embedding, simpan ke *vector database*, semua dalam satu request HTTP yang sama. Pamer ke teman, keliatan keren.\n\nBegitu aplikasinya ketemu pengguna beneran, masalahnya datang dari arah yang di demo belum kelihatan. Ada yang upload laporan keuangan 200 halaman penuh tabel. Parsing, chunking, dan panggilan API embedding-nya makan puluhan detik. Load balancer menyerah duluan dengan `504 Gateway Timeout`. Yang lebih nyebelin lagi: proses parsing yang rakus CPU menyandera thread server. Kueri chat pengguna lain ikut macet.\n\nTerus gimana cara bikin RAG yang tahan banting, tanpa harus nyewa klaster Kubernetes yang ribet dan mahal? Jawabannya bukan mesin yang lebih besar, tapi memisahkan dua beban yang sifatnya memang beda.\n\n## Dua jalur yang sifatnya kebalikan\n\nDi dalam sistem RAG ada dua beban kerja yang berlawanan karakternya:\n\n- *Jalur kueri*: pengguna bertanya, kueri di-*embed* (di bawah 100 ms), *vector search* (di bawah 50 ms), LLM men-*streaming* jawaban. Sifatnya *I\u002FO-bound* dan minta latensi serendah mungkin.\n- *Jalur ingestion*: pengguna upload dokumen, parsing layout, *chunking*, ratusan embedding dibikin, masuk *index*. Sifatnya *CPU-bound*, makan detik sampai menit.\n\nKalau dua jalur ini berbagi CPU di *service* yang sama, uptime-mu diserahkan ke pengguna yang paling hobi upload file gede. Aturan emasnya satu: jalur ingestion **wajib** dipisah jadi *worker* asinkron berbasis *queue*.\n\n## Kenapa Cloud Run dan Cloud Tasks\n\nBanyak tim langsung pasang Kafka atau RabbitMQ di VM sendiri buat ngatur *background worker*. Sah-sah aja sih, tapi buat tim kecil itu beban operasionalnya kebablasan. Duet serverless di GCP menyelesaikan masalah yang sama tanpa server yang harus dirawat:\n\n- *Cloud Run*: jalanin container apa pun, *scale-to-zero* saat nggak ada trafik (saat sepi, biayanya nol 💸), dan bisa *scale-out* ke ratusan instance otomatis waktu *queue* membludak 🚀\n- *Cloud Tasks*: *queue* yang dikelola penuh, dengan *rate limiting* bawaan (misalnya maksimal 10 task paralel biar kuota TPM\u002FRPM provider embedding nggak jebol), *retry* dengan *exponential backoff* (bawaannya `max_attempts` 100, *backoff* sampai 3600 detik), dan deduplikasi task. Yang terakhir ini ada syaratnya, nanti kita bahas.\n\n## Perjalanan satu dokumen, dari upload sampai siap\n\nBegini siklus hidup sebuah dokumen kalau kita telusuri di RAG yang sudah matang:\n\nPertama, upload langsung ke *storage*. File 50 MB nggak lewat backend. Backend cuma bikin GCS *signed URL*, lalu browser upload file binernya langsung ke *bucket*. Bandwidth dan memori server API aman.\n\nKedua, *enqueue* lalu jawab cepat. Setelah file mendarat di GCS, backend membuat task ke Cloud Tasks dengan payload metadata (`document_id`, `gcs_uri`). Lalu dia langsung membalas HTTP `202 Accepted` ke pengguna. Kueri chat pengguna lain nggak kena imbas apa pun.\n\nKetiga, *worker* jalan di container terpisah. Cloud Tasks mengirim HTTP POST ke endpoint *worker* di service Cloud Run yang lain. Langkahnya satu *chain*: unduh file, parsing, *chunking*, *batch embedding*. Lalu tulis ke *vector store* (Qdrant, Vertex AI Vector Search, atau pgvector), update status dokumen jadi `READY`, dan kabari pengguna lewat *webhook* atau *websocket*.\n\n## Kodenya, bagian yang bisa kupastikan jalan\n\nSekarang kita lihat kodenya. Ini bagian *enqueue*-nya, pakai `google-cloud-tasks` 2.26.0:\n\n```python\nfrom google.cloud import tasks_v2\nimport json\nimport time\nfrom fastapi import FastAPI, status\n\napp = FastAPI()\ntasks_client = tasks_v2.CloudTasksClient()\n\nPROJECT_ID = \"my-gcp-ai-project\"\nLOCATION_ID = \"asia-southeast2\"  # region Jakarta\nQUEUE_ID = \"document-ingestion-queue\"\n\nqueue_path = tasks_client.queue_path(PROJECT_ID, LOCATION_ID, QUEUE_ID)\n\n@app.post(\"\u002Fapi\u002Fv1\u002Fdocuments\u002Fprocess\", status_code=status.HTTP_202_ACCEPTED)\nasync def enqueue_document_processing(doc_id: str, gcs_uri: str):\n    # 1. Siapkan payload task\n    payload = {\n        \"document_id\": doc_id,\n        \"gcs_uri\": gcs_uri\n    }\n\n    # 2. Buat task HTTP target ke worker Cloud Run\n    task = {\n        # Nama eksplisit = deduplikasi: nama sama ditolak ALREADY_EXISTS\n        \"name\": f\"{queue_path}\u002Ftasks\u002F{doc_id}-{int(time.time() \u002F\u002F 60)}\",\n        \"http_request\": {\n            \"http_method\": tasks_v2.HttpMethod.POST,\n            \"url\": \"https:\u002F\u002Frag-worker-xyz-as.a.run.app\u002Finternal\u002Ftasks\u002Fingest\",\n            \"headers\": {\"Content-Type\": \"application\u002Fjson\"},\n            \"body\": json.dumps(payload).encode(),\n            # OIDC token untuk otentikasi aman antar-layanan di GCP!\n            \"oidc_token\": {\n                \"service_account_email\": \"tasks-invoker@my-gcp-ai-project.iam.gserviceaccount.com\"\n            }\n        }\n    }\n\n    # 3. Masukkan task ke queue Cloud Tasks\n    response = tasks_client.create_task(request={\"parent\": queue_path, \"task\": task})\n\n    return {\n        \"status\": \"queued\",\n        \"message\": \"Dokumen berhasil dimasukkan ke antrean pemrosesan asinkron.\",\n        \"task_id\": response.name\n    }\n```\n\nYang bisa kupastikan tanpa kredensial GCP: bentuk request-nya valid. `queue_path` kebentuk `projects\u002Fmy-gcp-ai-project\u002Flocations\u002Fasia-southeast2\u002Fqueues\u002Fdocument-ingestion-queue`, dan objek task-nya terserialisasi benar jadi `httpMethod: POST`, `oidcToken` dengan *service account* yang benar, dan *body* JSON yang utuh. `create_task` yang beneran mengirim ke *queue* butuh kredensial, jadi bagian itu sengaja nggak kujalankan di sini.\n\nDua catatan tambahan buat kode ini. Pertama, `oidc_token` di task itu tuh yang bikin *worker* Cloud Run boleh diproteksi, selama *service account*-nya pegang peran `run.invoker`. Tapi *ingress internal* ada syaratnya: Cloud Tasks baru dihitung internal kalau project-nya sama (atau satu perimeter VPC-SC) dan kamu pakai URL bawaan `run.app`. Kalau URL bawaannya kamu matikan atau kamu pakai *custom domain*, pemanggilannya berhenti jalan. Jadi buat *worker* privat, yang jadi tumpuan itu wajib autentikasinya, bukan cuma *ingress internal*-nya. Kedua, Cloud Tasks menganggap task berhasil kalau *worker* membalas 2xx sebelum *timeout* bawaan 10 menit (maksimal bisa dinaikkan ke 30). Kalau parsing PDF 200 halaman-mu lebih lama dari itu, task-nya dianggap gagal. Dia di-retry, padahal pekerjaannya belum tentu salah. Angka timeout ini wajib kamu sesuaikan dengan beban parsing-mu.\n\n## Dua detail yang nggak kelihatan di diagram\n\nDiagram arsitektur selalu kelihatan rapi, tapi dua hal ini nih yang biasanya baru ketahuan setelah tumbang:\n\n**Cloud Tasks itu at-least-once, bukan exactly-once.** Dokumentasi resminya bilang terus terang: dalam keadaan langka, satu task bisa dieksekusi lebih dari sekali, dan kamu sendiri yang harus memastikan pengulangan itu aman. Artinya kamu wajib bikin *worker*-nya **idempoten**: task yang sama masuk dua kali, hasilnya harus tetap sama. Ini prinsip yang sama dengan *checkpoint* idempoten di sistem orkestrasi *agent*, cuma beda konteks.\n\n**Deduplikasi cuma berlaku untuk task bernama eksplisit.** Tanpa nama, task yang sama bisa kekirim dua kali kalau sisi pemanggil me-retry `create_task`. Dengan nama, Cloud Tasks menolak duplikatnya dengan error `ALREADY_EXISTS`. Tapi ada jebakannya. Nama task diingat sampai 24 jam setelah task-nya selesai. Bahkan 9 hari kalau *queue*-nya dibikin dari `queue.yaml`. Jadi kalau namanya cuma `document_id`, pengguna yang upload ulang dokumen yang sama dalam 24 jam bakal ditolak, bukan di-*dedup*. Dari luar kelihatannya seperti *queue*-nya rusak. Karena itu di kode di atas namanya `{document_id}-{epoch_menit}`: upload yang sama tetap ke-*dedup* dalam menit yang sama, tapi upload ulang di hari berikutnya tetap jalan.\n\n## Cold start: satu angka per service\n\nTantangan khas serverless itu *cold start*, apalagi kalau container-nya memuat pustaka parsing yang berat. Obatnya beda-beda per service:\n\n- **Layanan kueri**: pasang `min-instances: 1`. Selalu ada satu instance hangat, jadi pengguna pertama nggak pernah kena jeda beberapa detik. Konsekuensinya instance idle itu tetap dibayar dengan tarif idle yang lebih murah (di mode *request-based billing*, yang jadi bawaan; di mode *instance-based* kamu bayar tarif penuh selama instance-nya hidup).\n- **Worker ingestion**: biarkan di `min-instances: 0`. Nggak ada pengguna yang nunggu di layar, jadi aman *scale-to-zero*, dan biayanya nol waktu sepi.\n- **Image-nya dirampingkan**: *multi-stage build*, cache pip nggak ikut, *base image* `slim` (misalnya `python:3.13-slim`), supaya waktu narik image-nya tetap sebentar.\n\n## Yang bikin RAG kuat bukan modelnya\n\nSistem RAG yang bertahan di produksi bukan yang LLM-nya paling gede, tapi yang disiplin soal siapa boleh pakai CPU kapan. Pisah jalur cepat dan jalur berat, percayakan *queue*-nya ke layanan yang emang dibuat buat itu, dan biaya yang kamu bayar cuma komputasi pas dokumen beneran diproses.\n\nKalau kueri chat-mu mulai lemot tiap ada yang upload dokumen, yang disalahkan bukan model embedding-nya, tapi arsitekturnya. Pisah jalurnya dulu, sisanya tinggal urusan angka. Semoga 504-nya pamit 😄\n","- Jalur kueri dan jalur ingestion punya karakter beban yang kebalikan, jadi ingestion wajib dipindah ke *worker* asinkron berbasis *queue*.\n- Upload lewat GCS *signed URL* menjaga backend tetap ringan, karena file puluhan megabyte nggak pernah lewat memori server API.\n- Cloud Tasks mengirim task secara *at-least-once* dan deduplikasinya cuma berlaku untuk task bernama eksplisit, jadi *worker* ingestion wajib **idempoten**.\n- `min-instances: 1` di layanan kueri dan `0` di *worker* menyelesaikan masalah *cold start* tanpa membayar instance yang menganggur.","\u003Cp>Bikin RAG di masa hackathon itu terasa instan. Satu container FastAPI, terima file PDF, ekstrak teks, bikin embedding, simpan ke \u003Cem>vector database\u003C\u002Fem>, semua dalam satu request HTTP yang sama. Pamer ke teman, keliatan keren.\u003C\u002Fp>\n\u003Cp>Begitu aplikasinya ketemu pengguna beneran, masalahnya datang dari arah yang di demo belum kelihatan. Ada yang upload laporan keuangan 200 halaman penuh tabel. Parsing, chunking, dan panggilan API embedding-nya makan puluhan detik. Load balancer menyerah duluan dengan \u003Ccode>504 Gateway Timeout\u003C\u002Fcode>. Yang lebih nyebelin lagi: proses parsing yang rakus CPU menyandera thread server. Kueri chat pengguna lain ikut macet.\u003C\u002Fp>\n\u003Cp>Terus gimana cara bikin RAG yang tahan banting, tanpa harus nyewa klaster Kubernetes yang ribet dan mahal? Jawabannya bukan mesin yang lebih besar, tapi memisahkan dua beban yang sifatnya memang beda.\u003C\u002Fp>\n\u003Ch2 id=\"dua-jalur-yang-sifatnya-kebalikan\">Dua jalur yang sifatnya kebalikan\u003C\u002Fh2>\n\u003Cp>Di dalam sistem RAG ada dua beban kerja yang berlawanan karakternya:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cem>Jalur kueri\u003C\u002Fem>: pengguna bertanya, kueri di-\u003Cem>embed\u003C\u002Fem> (di bawah 100 ms), \u003Cem>vector search\u003C\u002Fem> (di bawah 50 ms), LLM men-\u003Cem>streaming\u003C\u002Fem> jawaban. Sifatnya \u003Cem>I\u002FO-bound\u003C\u002Fem> dan minta latensi serendah mungkin.\u003C\u002Fli>\n\u003Cli>\u003Cem>Jalur ingestion\u003C\u002Fem>: pengguna upload dokumen, parsing layout, \u003Cem>chunking\u003C\u002Fem>, ratusan embedding dibikin, masuk \u003Cem>index\u003C\u002Fem>. Sifatnya \u003Cem>CPU-bound\u003C\u002Fem>, makan detik sampai menit.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Kalau dua jalur ini berbagi CPU di \u003Cem>service\u003C\u002Fem> yang sama, uptime-mu diserahkan ke pengguna yang paling hobi upload file gede. Aturan emasnya satu: jalur ingestion \u003Cstrong>wajib\u003C\u002Fstrong> dipisah jadi \u003Cem>worker\u003C\u002Fem> asinkron berbasis \u003Cem>queue\u003C\u002Fem>.\u003C\u002Fp>\n\u003Ch2 id=\"kenapa-cloud-run-dan-cloud-tasks\">Kenapa Cloud Run dan Cloud Tasks\u003C\u002Fh2>\n\u003Cp>Banyak tim langsung pasang Kafka atau RabbitMQ di VM sendiri buat ngatur \u003Cem>background worker\u003C\u002Fem>. Sah-sah aja sih, tapi buat tim kecil itu beban operasionalnya kebablasan. Duet serverless di GCP menyelesaikan masalah yang sama tanpa server yang harus dirawat:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cem>Cloud Run\u003C\u002Fem>: jalanin container apa pun, \u003Cem>scale-to-zero\u003C\u002Fem> saat nggak ada trafik (saat sepi, biayanya nol 💸), dan bisa \u003Cem>scale-out\u003C\u002Fem> ke ratusan instance otomatis waktu \u003Cem>queue\u003C\u002Fem> membludak 🚀\u003C\u002Fli>\n\u003Cli>\u003Cem>Cloud Tasks\u003C\u002Fem>: \u003Cem>queue\u003C\u002Fem> yang dikelola penuh, dengan \u003Cem>rate limiting\u003C\u002Fem> bawaan (misalnya maksimal 10 task paralel biar kuota TPM\u002FRPM provider embedding nggak jebol), \u003Cem>retry\u003C\u002Fem> dengan \u003Cem>exponential backoff\u003C\u002Fem> (bawaannya \u003Ccode>max_attempts\u003C\u002Fcode> 100, \u003Cem>backoff\u003C\u002Fem> sampai 3600 detik), dan deduplikasi task. Yang terakhir ini ada syaratnya, nanti kita bahas.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2 id=\"perjalanan-satu-dokumen-dari-upload-sampai-siap\">Perjalanan satu dokumen, dari upload sampai siap\u003C\u002Fh2>\n\u003Cp>Begini siklus hidup sebuah dokumen kalau kita telusuri di RAG yang sudah matang:\u003C\u002Fp>\n\u003Cp>Pertama, upload langsung ke \u003Cem>storage\u003C\u002Fem>. File 50 MB nggak lewat backend. Backend cuma bikin GCS \u003Cem>signed URL\u003C\u002Fem>, lalu browser upload file binernya langsung ke \u003Cem>bucket\u003C\u002Fem>. Bandwidth dan memori server API aman.\u003C\u002Fp>\n\u003Cp>Kedua, \u003Cem>enqueue\u003C\u002Fem> lalu jawab cepat. Setelah file mendarat di GCS, backend membuat task ke Cloud Tasks dengan payload metadata (\u003Ccode>document_id\u003C\u002Fcode>, \u003Ccode>gcs_uri\u003C\u002Fcode>). Lalu dia langsung membalas HTTP \u003Ccode>202 Accepted\u003C\u002Fcode> ke pengguna. Kueri chat pengguna lain nggak kena imbas apa pun.\u003C\u002Fp>\n\u003Cp>Ketiga, \u003Cem>worker\u003C\u002Fem> jalan di container terpisah. Cloud Tasks mengirim HTTP POST ke endpoint \u003Cem>worker\u003C\u002Fem> di service Cloud Run yang lain. Langkahnya satu \u003Cem>chain\u003C\u002Fem>: unduh file, parsing, \u003Cem>chunking\u003C\u002Fem>, \u003Cem>batch embedding\u003C\u002Fem>. Lalu tulis ke \u003Cem>vector store\u003C\u002Fem> (Qdrant, Vertex AI Vector Search, atau pgvector), update status dokumen jadi \u003Ccode>READY\u003C\u002Fcode>, dan kabari pengguna lewat \u003Cem>webhook\u003C\u002Fem> atau \u003Cem>websocket\u003C\u002Fem>.\u003C\u002Fp>\n\u003Ch2 id=\"kodenya-bagian-yang-bisa-kupastikan-jalan\">Kodenya, bagian yang bisa kupastikan jalan\u003C\u002Fh2>\n\u003Cp>Sekarang kita lihat kodenya. Ini bagian \u003Cem>enqueue\u003C\u002Fem>-nya, pakai \u003Ccode>google-cloud-tasks\u003C\u002Fcode> 2.26.0:\u003C\u002Fp>\n\n\u003Cdiv class=\"code-block-wrapper my-6 not-prose rounded-xl border-2 border-black dark:border-gray-600 shadow-neo overflow-hidden bg-[#24292e] text-[#e1e4e8] transition-all\">\n  \u003Cdiv class=\"code-block-header flex items-center justify-between px-4 py-2.5 bg-[#1f2428] border-b-2 border-black dark:border-gray-600 font-mono text-xs font-bold text-gray-300 select-none\">\n    \u003Cdiv class=\"flex items-center gap-2\">\n      \u003Cspan class=\"w-3 h-3 rounded-full bg-[#ff5f56] border border-black\u002F40 inline-block\">\u003C\u002Fspan>\n      \u003Cspan class=\"w-3 h-3 rounded-full bg-[#ffbd2e] border border-black\u002F40 inline-block\">\u003C\u002Fspan>\n      \u003Cspan class=\"w-3 h-3 rounded-full bg-[#27c93f] border border-black\u002F40 inline-block\">\u003C\u002Fspan>\n      \u003Cspan class=\"ml-2 font-mono font-bold text-xs uppercase tracking-wider text-gray-300\">PYTHON\u003C\u002Fspan>\n    \u003C\u002Fdiv>\n    \u003Cbutton type=\"button\" class=\"copy-code-btn flex items-center gap-1.5 px-3 py-1 text-xs font-bold bg-[#2f363d] text-gray-200 border-2 border-black dark:border-gray-500 rounded-lg shadow-neo-sm hover:translate-x-0.5 hover:translate-y-0.5 hover:shadow-none hover:bg-gray-700 transition-all cursor-pointer\" data-code=\"from%20google.cloud%20import%20tasks_v2%0Aimport%20json%0Aimport%20time%0Afrom%20fastapi%20import%20FastAPI%2C%20status%0A%0Aapp%20%3D%20FastAPI()%0Atasks_client%20%3D%20tasks_v2.CloudTasksClient()%0A%0APROJECT_ID%20%3D%20%22my-gcp-ai-project%22%0ALOCATION_ID%20%3D%20%22asia-southeast2%22%20%20%23%20region%20Jakarta%0AQUEUE_ID%20%3D%20%22document-ingestion-queue%22%0A%0Aqueue_path%20%3D%20tasks_client.queue_path(PROJECT_ID%2C%20LOCATION_ID%2C%20QUEUE_ID)%0A%0A%40app.post(%22%2Fapi%2Fv1%2Fdocuments%2Fprocess%22%2C%20status_code%3Dstatus.HTTP_202_ACCEPTED)%0Aasync%20def%20enqueue_document_processing(doc_id%3A%20str%2C%20gcs_uri%3A%20str)%3A%0A%20%20%20%20%23%201.%20Siapkan%20payload%20task%0A%20%20%20%20payload%20%3D%20%7B%0A%20%20%20%20%20%20%20%20%22document_id%22%3A%20doc_id%2C%0A%20%20%20%20%20%20%20%20%22gcs_uri%22%3A%20gcs_uri%0A%20%20%20%20%7D%0A%0A%20%20%20%20%23%202.%20Buat%20task%20HTTP%20target%20ke%20worker%20Cloud%20Run%0A%20%20%20%20task%20%3D%20%7B%0A%20%20%20%20%20%20%20%20%23%20Nama%20eksplisit%20%3D%20deduplikasi%3A%20nama%20sama%20ditolak%20ALREADY_EXISTS%0A%20%20%20%20%20%20%20%20%22name%22%3A%20f%22%7Bqueue_path%7D%2Ftasks%2F%7Bdoc_id%7D-%7Bint(time.time()%20%2F%2F%2060)%7D%22%2C%0A%20%20%20%20%20%20%20%20%22http_request%22%3A%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%22http_method%22%3A%20tasks_v2.HttpMethod.POST%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%22url%22%3A%20%22https%3A%2F%2Frag-worker-xyz-as.a.run.app%2Finternal%2Ftasks%2Fingest%22%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%22headers%22%3A%20%7B%22Content-Type%22%3A%20%22application%2Fjson%22%7D%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%22body%22%3A%20json.dumps(payload).encode()%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%23%20OIDC%20token%20untuk%20otentikasi%20aman%20antar-layanan%20di%20GCP!%0A%20%20%20%20%20%20%20%20%20%20%20%20%22oidc_token%22%3A%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%22service_account_email%22%3A%20%22tasks-invoker%40my-gcp-ai-project.iam.gserviceaccount.com%22%0A%20%20%20%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%0A%20%20%20%20%23%203.%20Masukkan%20task%20ke%20queue%20Cloud%20Tasks%0A%20%20%20%20response%20%3D%20tasks_client.create_task(request%3D%7B%22parent%22%3A%20queue_path%2C%20%22task%22%3A%20task%7D)%0A%0A%20%20%20%20return%20%7B%0A%20%20%20%20%20%20%20%20%22status%22%3A%20%22queued%22%2C%0A%20%20%20%20%20%20%20%20%22message%22%3A%20%22Dokumen%20berhasil%20dimasukkan%20ke%20antrean%20pemrosesan%20asinkron.%22%2C%0A%20%20%20%20%20%20%20%20%22task_id%22%3A%20response.name%0A%20%20%20%20%7D\" title=\"Salin kode\">\n      \u003Csvg class=\"w-3.5 h-3.5 copy-icon-svg\" fill=\"none\" stroke=\"currentColor\" viewBox=\"0 0 24 24\">\n        \u003Cpath stroke-linecap=\"round\" stroke-linejoin=\"round\" stroke-width=\"2\" d=\"M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z\">\u003C\u002Fpath>\n      \u003C\u002Fsvg>\n      \u003Cspan class=\"copy-text\">Copy\u003C\u002Fspan>\n    \u003C\u002Fbutton>\n  \u003C\u002Fdiv>\n  \u003Cdiv class=\"code-block-content p-4 overflow-x-auto text-sm font-mono leading-relaxed bg-[#24292e]\">\n    \u003Cpre class=\"shiki github-dark\" style=\"background-color:#24292e;color:#e1e4e8\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#F97583\">from\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> google.cloud \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">import\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> tasks_v2\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#F97583\">import\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> json\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#F97583\">import\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> time\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#F97583\">from\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> fastapi \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">import\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> FastAPI, status\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">app \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> FastAPI()\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">tasks_client \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> tasks_v2.CloudTasksClient()\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#79B8FF\">PROJECT_ID\u003C\u002Fspan>\u003Cspan style=\"color:#F97583\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\"> \"my-gcp-ai-project\"\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#79B8FF\">LOCATION_ID\u003C\u002Fspan>\u003Cspan style=\"color:#F97583\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\"> \"asia-southeast2\"\u003C\u002Fspan>\u003Cspan style=\"color:#6A737D\">  # region Jakarta\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#79B8FF\">QUEUE_ID\u003C\u002Fspan>\u003Cspan style=\"color:#F97583\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\"> \"document-ingestion-queue\"\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">queue_path \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> tasks_client.queue_path(\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">PROJECT_ID\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">, \u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">LOCATION_ID\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">, \u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">QUEUE_ID\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">)\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#B392F0\">@app.post\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">(\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"\u002Fapi\u002Fv1\u002Fdocuments\u002Fprocess\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">, \u003C\u002Fspan>\u003Cspan style=\"color:#FFAB70\">status_code\u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">status.\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">HTTP_202_ACCEPTED\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">)\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#F97583\">async\u003C\u002Fspan>\u003Cspan style=\"color:#F97583\"> def\u003C\u002Fspan>\u003Cspan style=\"color:#B392F0\"> enqueue_document_processing\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">(doc_id: \u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">str\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">, gcs_uri: \u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">str\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">):\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6A737D\">    # 1. Siapkan payload task\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">    payload \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">        \"document_id\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: doc_id,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">        \"gcs_uri\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: gcs_uri\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">    }\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6A737D\">    # 2. Buat task HTTP target ke worker Cloud Run\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">    task \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6A737D\">        # Nama eksplisit = deduplikasi: nama sama ditolak ALREADY_EXISTS\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">        \"name\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">f\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">{\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">queue_path\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">}\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\u002Ftasks\u002F\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">{\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">doc_id\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">}\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">-\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">{int\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">(time.time() \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">\u002F\u002F\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\"> 60\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">)\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">}\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">        \"http_request\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">            \"http_method\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: tasks_v2.HttpMethod.\u003C\u002Fspan>\u003Cspan style=\"color:#79B8FF\">POST\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">            \"url\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: \u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"https:\u002F\u002Frag-worker-xyz-as.a.run.app\u002Finternal\u002Ftasks\u002Fingest\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">            \"headers\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: {\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"Content-Type\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: \u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"application\u002Fjson\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">},\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">            \"body\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: json.dumps(payload).encode(),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6A737D\">            # OIDC token untuk otentikasi aman antar-layanan di GCP!\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">            \"oidc_token\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">                \"service_account_email\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: \u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"tasks-invoker@my-gcp-ai-project.iam.gserviceaccount.com\"\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">            }\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">        }\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">    }\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6A737D\">    # 3. Masukkan task ke queue Cloud Tasks\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">    response \u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> tasks_client.create_task(\u003C\u002Fspan>\u003Cspan style=\"color:#FFAB70\">request\u003C\u002Fspan>\u003Cspan style=\"color:#F97583\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">{\u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"parent\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: queue_path, \u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"task\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: task})\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#F97583\">    return\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">        \"status\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: \u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"queued\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">        \"message\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: \u003C\u002Fspan>\u003Cspan style=\"color:#9ECBFF\">\"Dokumen berhasil dimasukkan ke antrean pemrosesan asinkron.\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#9ECBFF\">        \"task_id\"\u003C\u002Fspan>\u003Cspan style=\"color:#E1E4E8\">: response.name\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E1E4E8\">    }\u003C\u002Fspan>\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n  \u003C\u002Fdiv>\n\u003C\u002Fdiv>\u003Cp>Yang bisa kupastikan tanpa kredensial GCP: bentuk request-nya valid. \u003Ccode>queue_path\u003C\u002Fcode> kebentuk \u003Ccode>projects\u002Fmy-gcp-ai-project\u002Flocations\u002Fasia-southeast2\u002Fqueues\u002Fdocument-ingestion-queue\u003C\u002Fcode>, dan objek task-nya terserialisasi benar jadi \u003Ccode>httpMethod: POST\u003C\u002Fcode>, \u003Ccode>oidcToken\u003C\u002Fcode> dengan \u003Cem>service account\u003C\u002Fem> yang benar, dan \u003Cem>body\u003C\u002Fem> JSON yang utuh. \u003Ccode>create_task\u003C\u002Fcode> yang beneran mengirim ke \u003Cem>queue\u003C\u002Fem> butuh kredensial, jadi bagian itu sengaja nggak kujalankan di sini.\u003C\u002Fp>\n\u003Cp>Dua catatan tambahan buat kode ini. Pertama, \u003Ccode>oidc_token\u003C\u002Fcode> di task itu tuh yang bikin \u003Cem>worker\u003C\u002Fem> Cloud Run boleh diproteksi, selama \u003Cem>service account\u003C\u002Fem>-nya pegang peran \u003Ccode>run.invoker\u003C\u002Fcode>. Tapi \u003Cem>ingress internal\u003C\u002Fem> ada syaratnya: Cloud Tasks baru dihitung internal kalau project-nya sama (atau satu perimeter VPC-SC) dan kamu pakai URL bawaan \u003Ccode>run.app\u003C\u002Fcode>. Kalau URL bawaannya kamu matikan atau kamu pakai \u003Cem>custom domain\u003C\u002Fem>, pemanggilannya berhenti jalan. Jadi buat \u003Cem>worker\u003C\u002Fem> privat, yang jadi tumpuan itu wajib autentikasinya, bukan cuma \u003Cem>ingress internal\u003C\u002Fem>-nya. Kedua, Cloud Tasks menganggap task berhasil kalau \u003Cem>worker\u003C\u002Fem> membalas 2xx sebelum \u003Cem>timeout\u003C\u002Fem> bawaan 10 menit (maksimal bisa dinaikkan ke 30). Kalau parsing PDF 200 halaman-mu lebih lama dari itu, task-nya dianggap gagal. Dia di-retry, padahal pekerjaannya belum tentu salah. Angka timeout ini wajib kamu sesuaikan dengan beban parsing-mu.\u003C\u002Fp>\n\u003Ch2 id=\"dua-detail-yang-nggak-kelihatan-di-diagram\">Dua detail yang nggak kelihatan di diagram\u003C\u002Fh2>\n\u003Cp>Diagram arsitektur selalu kelihatan rapi, tapi dua hal ini nih yang biasanya baru ketahuan setelah tumbang:\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Cloud Tasks itu at-least-once, bukan exactly-once.\u003C\u002Fstrong> Dokumentasi resminya bilang terus terang: dalam keadaan langka, satu task bisa dieksekusi lebih dari sekali, dan kamu sendiri yang harus memastikan pengulangan itu aman. Artinya kamu wajib bikin \u003Cem>worker\u003C\u002Fem>-nya \u003Cstrong>idempoten\u003C\u002Fstrong>: task yang sama masuk dua kali, hasilnya harus tetap sama. Ini prinsip yang sama dengan \u003Cem>checkpoint\u003C\u002Fem> idempoten di sistem orkestrasi \u003Cem>agent\u003C\u002Fem>, cuma beda konteks.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Deduplikasi cuma berlaku untuk task bernama eksplisit.\u003C\u002Fstrong> Tanpa nama, task yang sama bisa kekirim dua kali kalau sisi pemanggil me-retry \u003Ccode>create_task\u003C\u002Fcode>. Dengan nama, Cloud Tasks menolak duplikatnya dengan error \u003Ccode>ALREADY_EXISTS\u003C\u002Fcode>. Tapi ada jebakannya. Nama task diingat sampai 24 jam setelah task-nya selesai. Bahkan 9 hari kalau \u003Cem>queue\u003C\u002Fem>-nya dibikin dari \u003Ccode>queue.yaml\u003C\u002Fcode>. Jadi kalau namanya cuma \u003Ccode>document_id\u003C\u002Fcode>, pengguna yang upload ulang dokumen yang sama dalam 24 jam bakal ditolak, bukan di-\u003Cem>dedup\u003C\u002Fem>. Dari luar kelihatannya seperti \u003Cem>queue\u003C\u002Fem>-nya rusak. Karena itu di kode di atas namanya \u003Ccode>{document_id}-{epoch_menit}\u003C\u002Fcode>: upload yang sama tetap ke-\u003Cem>dedup\u003C\u002Fem> dalam menit yang sama, tapi upload ulang di hari berikutnya tetap jalan.\u003C\u002Fp>\n\u003Ch2 id=\"cold-start-satu-angka-per-service\">Cold start: satu angka per service\u003C\u002Fh2>\n\u003Cp>Tantangan khas serverless itu \u003Cem>cold start\u003C\u002Fem>, apalagi kalau container-nya memuat pustaka parsing yang berat. Obatnya beda-beda per service:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cstrong>Layanan kueri\u003C\u002Fstrong>: pasang \u003Ccode>min-instances: 1\u003C\u002Fcode>. Selalu ada satu instance hangat, jadi pengguna pertama nggak pernah kena jeda beberapa detik. Konsekuensinya instance idle itu tetap dibayar dengan tarif idle yang lebih murah (di mode \u003Cem>request-based billing\u003C\u002Fem>, yang jadi bawaan; di mode \u003Cem>instance-based\u003C\u002Fem> kamu bayar tarif penuh selama instance-nya hidup).\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Worker ingestion\u003C\u002Fstrong>: biarkan di \u003Ccode>min-instances: 0\u003C\u002Fcode>. Nggak ada pengguna yang nunggu di layar, jadi aman \u003Cem>scale-to-zero\u003C\u002Fem>, dan biayanya nol waktu sepi.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Image-nya dirampingkan\u003C\u002Fstrong>: \u003Cem>multi-stage build\u003C\u002Fem>, cache pip nggak ikut, \u003Cem>base image\u003C\u002Fem> \u003Ccode>slim\u003C\u002Fcode> (misalnya \u003Ccode>python:3.13-slim\u003C\u002Fcode>), supaya waktu narik image-nya tetap sebentar.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2 id=\"yang-bikin-rag-kuat-bukan-modelnya\">Yang bikin RAG kuat bukan modelnya\u003C\u002Fh2>\n\u003Cp>Sistem RAG yang bertahan di produksi bukan yang LLM-nya paling gede, tapi yang disiplin soal siapa boleh pakai CPU kapan. Pisah jalur cepat dan jalur berat, percayakan \u003Cem>queue\u003C\u002Fem>-nya ke layanan yang emang dibuat buat itu, dan biaya yang kamu bayar cuma komputasi pas dokumen beneran diproses.\u003C\u002Fp>\n\u003Cp>Kalau kueri chat-mu mulai lemot tiap ada yang upload dokumen, yang disalahkan bukan model embedding-nya, tapi arsitekturnya. Pisah jalurnya dulu, sisanya tinggal urusan angka. Semoga 504-nya pamit 😄\u003C\u002Fp>\n","\u003Cul>\n\u003Cli>Jalur kueri dan jalur ingestion punya karakter beban yang kebalikan, jadi ingestion wajib dipindah ke \u003Cem>worker\u003C\u002Fem> asinkron berbasis \u003Cem>queue\u003C\u002Fem>.\u003C\u002Fli>\n\u003Cli>Upload lewat GCS \u003Cem>signed URL\u003C\u002Fem> menjaga backend tetap ringan, karena file puluhan megabyte nggak pernah lewat memori server API.\u003C\u002Fli>\n\u003Cli>Cloud Tasks mengirim task secara \u003Cem>at-least-once\u003C\u002Fem> dan deduplikasinya cuma berlaku untuk task bernama eksplisit, jadi \u003Cem>worker\u003C\u002Fem> ingestion wajib \u003Cstrong>idempoten\u003C\u002Fstrong>.\u003C\u002Fli>\n\u003Cli>\u003Ccode>min-instances: 1\u003C\u002Fcode> di layanan kueri dan \u003Ccode>0\u003C\u002Fcode> di \u003Cem>worker\u003C\u002Fem> menyelesaikan masalah \u003Cem>cold start\u003C\u002Fem> tanpa membayar instance yang menganggur.\u003C\u002Fli>\n\u003C\u002Ful>\n",[268,285,298,311,324,329],{"id":269,"title":270,"title_en":21,"slug":271,"excerpt":272,"excerpt_en":21,"has_translation":15,"published_at":273,"reading_time":213,"hidden_from_listing":15,"tags":274,"thumbnail_image":21,"og_image":276,"series":277,"authors":282},133,"Python Strings: the Badge That Only Worked for One Name","python-strings","The badge script printed the right name for the person who wrote it and the wrong name for everybody else. Slicing counts characters, and a name is words.","2026-10-28T13:00:00.000Z",[184,275],"string","\u002Fog\u002Fpython-strings.png",{"id":278,"title":279,"title_en":280,"slug":281},30,"Koleksi dan String","Collections and Strings","python-collections",[283],{"authors_id":284},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":286,"title":287,"title_en":21,"slug":288,"excerpt":289,"excerpt_en":21,"has_translation":15,"published_at":290,"reading_time":213,"hidden_from_listing":15,"tags":291,"thumbnail_image":21,"og_image":293,"series":294,"authors":295},132,"Python Tuples: Why All Three Points Are the Same Point","python-tuples","Three points walk into a loop and come out identical. The list was never copied three times; the name was. A tuple makes that bug impossible.","2026-10-28T12:00:00.000Z",[184,292],"tuple","\u002Fog\u002Fpython-tuples.png",{"id":278,"title":279,"title_en":280,"slug":281},[296],{"authors_id":297},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":299,"title":300,"title_en":21,"slug":301,"excerpt":302,"excerpt_en":21,"has_translation":15,"published_at":303,"reading_time":213,"hidden_from_listing":15,"tags":304,"thumbnail_image":21,"og_image":306,"series":307,"authors":308},131,"Python Sets: the Duplicate That Keeps Coming Back","python-sets","A de-duplication loop that works on clean data leaves four tags where there should be two. The bug is case, and the fix is a `set`.","2026-10-28T11:00:00.000Z",[184,305],"set","\u002Fog\u002Fpython-sets.png",{"id":278,"title":279,"title_en":280,"slug":281},[309],{"authors_id":310},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":312,"title":313,"title_en":21,"slug":314,"excerpt":315,"excerpt_en":21,"has_translation":15,"published_at":316,"reading_time":213,"hidden_from_listing":15,"tags":317,"thumbnail_image":21,"og_image":319,"series":320,"authors":321},130,"Python Dictionaries: the Key That Isn't There","python-dictionary","A word counter dies the first time it meets a word twice. The fix is one line; what takes longer is knowing which of a dict's three lookups to reach for.","2026-10-28T10:00:00.000Z",[184,318],"dictionary","\u002Fog\u002Fpython-dictionary.png",{"id":278,"title":279,"title_en":280,"slug":281},[322],{"authors_id":323},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":74,"title":75,"title_en":76,"slug":77,"excerpt":78,"excerpt_en":79,"has_translation":12,"published_at":80,"reading_time":14,"hidden_from_listing":15,"tags":81,"thumbnail_image":21,"og_image":88,"series":325,"authors":326},{"id":90,"title":35,"title_en":36,"slug":37},[327],{"authors_id":328},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},{"id":226,"title":227,"title_en":228,"slug":229,"excerpt":230,"excerpt_en":231,"has_translation":12,"published_at":232,"reading_time":213,"hidden_from_listing":15,"tags":233,"thumbnail_image":21,"og_image":239,"series":330,"authors":331},{"id":90,"title":35,"title_en":36,"slug":37},[332],{"authors_id":333},{"id":94,"name":125,"slug":126,"role":127,"profile_picture":21},[335,338,341,344,347,350,353],{"text":336,"depth":111,"id":337},"Dua jalur yang sifatnya kebalikan","dua-jalur-yang-sifatnya-kebalikan",{"text":339,"depth":111,"id":340},"Kenapa Cloud Run dan Cloud Tasks","kenapa-cloud-run-dan-cloud-tasks",{"text":342,"depth":111,"id":343},"Perjalanan satu dokumen, dari upload sampai siap","perjalanan-satu-dokumen-dari-upload-sampai-siap",{"text":345,"depth":111,"id":346},"Kodenya, bagian yang bisa kupastikan jalan","kodenya-bagian-yang-bisa-kupastikan-jalan",{"text":348,"depth":111,"id":349},"Dua detail yang nggak kelihatan di diagram","dua-detail-yang-nggak-kelihatan-di-diagram",{"text":351,"depth":111,"id":352},"Cold start: satu angka per service","cold-start-satu-angka-per-service",{"text":354,"depth":111,"id":355},"Yang bikin RAG kuat bukan modelnya","yang-bikin-rag-kuat-bukan-modelnya",1793295835068]