{"id":100040416,"date":"2026-10-18T16:31:35","date_gmt":"2026-10-18T16:31:35","guid":{"rendered":"https:\/\/www.nbn.org.il\/jobboard\/?post_type=job_listing&p=100040416"},"modified":"2026-10-18T16:31:35","modified_gmt":"2026-10-18T16:31:35","slug":"senior-ai-inference-engineer","status":"publish","type":"job_listing","link":"https:\/\/www.nbn.org.il\/jobboard\/job\/senior-ai-inference-engineer\/","title":{"rendered":"Senior AI Inference Engineer"},"content":{"rendered":"

The Role<\/strong><\/p>\n

We need someone who can take a model, a set of GPUs, and a production requirement and determine how that model should actually run.<\/p>\n

Given a frontier open-weight model \u2014 dense or mixture-of-experts, tens to hundreds of billions of parameters \u2014 you should be able to determine the right precision, quantization, tensor\/pipeline\/expert parallelism, memory strategy, batching, concurrency, topology, and serving runtime. Then defend those decisions, measure the results, and operate the system in production.<\/p>\n

Beyond operating inference infrastructure, we want to turn that expertise into software. Decisions that currently require an experienced engineer should become reproducible, testable, and eventually automated.<\/p>\n

This is not a research position or an architecture-only role. You will benchmark, deploy, debug, tune, automate, and operate real inference systems with customers depending on them.<\/p>\n

What You Will Own<\/strong><\/p>\n

– Model placement, parallelism, precision, quantization, and memory strategy across single-GPU, multi-GPU, and multi-node deployments.
\n– Selection, evaluation, and operation of serving runtimes including vLLM, SGLang, TensorRT-LLM, and emerging alternatives.
\n– Evidence-based concurrency limits, latency and time-to-first-token (TTFT) targets, throughput expectations, and capacity commitments.
\n– Benchmarking methodology, tooling, reproducibility, and performance analysis.
\n– Production model lifecycle: loading, startup, health, upgrades, capacity management, and failure handling.
\n– Architecture and operational boundaries between multi-tenant and dedicated inference.
\n– Incident response and root-cause analysis across inference runtimes, GPUs, drivers, hosts, and networks.
\n– Software that automates inference configuration, placement, capacity planning, and deployment decisions.<\/p>\n

What Success Looks Like<\/strong><\/p>\n

Given a model, a hardware target, and a workload profile, you can quickly produce a deployment that is sensible, measurable, reproducible, and production-ready.<\/p>\n

You can answer which hardware can serve a model, how to distribute it across GPUs or nodes, which runtime and precision to use, what concurrency is sustainable, and how to balance latency, throughput, and cost.<\/p>\n

Then you help StackYak answer those questions programmatically instead of requiring an expert for every deployment.<\/p>\n

How We Work<\/strong><\/p>\n

– Small, senior team with direct access to the founders.
\n– Strong opinions, loosely held. Everyone participates in technical decisions.
\n– Shared responsibility for production reliability and on-call.
\n– Engineers move between design, implementation, debugging, and operations.
\n– We value demonstrated accomplishments over degrees, certifications, and credentials.
\n– We debate technical decisions, commit to a direction, and execute.
\n– Remote, distributed team working across time zones.
\n– We are hiring across inference, compute infrastructure, and networking. Boundaries are intentionally flexible.
\n– Early-stage startup environment with substantial ownership, fast execution, and evolving priorities.<\/p>\n

Compensation<\/strong><\/p>\n

Competitive compensation plus meaningful equity. Exact structure depends on location, engagement model, and experience. Employment versus contracting arrangements are determined based on location.<\/p>\n

Required Experience<\/strong><\/p>\n

We care about what you have already built and operated, not just what you know in theory. You should have direct experience with most of the following:<\/p>\n

– Operating large language models in production under real customer workloads.
\n– Serving large models across multiple GPUs and multiple nodes.
\n– Sizing models against available hardware, including GPU memory and topology constraints.
\n– Selecting precision and quantization strategies with measurable implications for quality, performance, and cost.
\n– Tuning batching, concurrency, and memory utilization beyond default configurations.
\n– Measuring and interpreting latency, TTFT, throughput, utilization, and inference cost.
\n– Working with NVIDIA and\/or AMD GPU inference environments.
\n– Building production Python software, infrastructure tooling, and automation.
\n– Debugging Linux and systems problems below the container boundary.
\n– Diagnosing inference failures originating in GPU hardware, drivers, hosts, networking, or other infrastructure rather than the serving runtime itself.
\n– Taking ownership of production systems, including operational reliability, troubleshooting, and incident response.<\/p>\n

Especially Valuable Experience<\/strong><\/p>\n

– Operating inference infrastructure at an AI company, inference provider, GPU cloud, hyperscaler, or substantial internal AI platform.
\n– Working across multiple GPU generations and vendors, with practical understanding of how hardware differences affect deployment decisions.
\n– Understanding distributed inference and the networking implications of multi-node serving.
\n– Benchmarking and comparing vLLM, SGLang, TensorRT-LLM, or other serving runtimes.
\n– Making and explaining architectural trade-offs rather than relying on vendor recipes or framework defaults.
\n– Building automated deployment systems, schedulers, placement engines, or capacity planners.
\n– Turning specialized operational knowledge into repeatable software and automation.
\n– Independently exploring, building, and testing systems beyond assigned work.<\/p>\n

This Role Is Probably Not a Fit If<\/strong><\/p>\n

– Your experience is primarily model training or ML research rather than production inference.
\n– You have deployed a model behind an API but have not worked directly with GPU memory, parallelism, topology, and concurrency.
\n– You treat default serving framework configurations as a complete architecture.
\n– You enjoy benchmarking but do not want responsibility for operating production systems.
\n– You prefer kernel optimization without interest in reliability, infrastructure, or productization.
\n– You prefer documenting recommendations over implementing and operating solutions.
\n– You want narrowly defined responsibilities or need an extended ramp-up before taking ownership.
\n– You would rather remain the person with specialized knowledge than build software that makes that knowledge broadly accessible.<\/p>\n

\r\nTell them you heard about the position from Nefesh B\u2019Nefesh.<\/span>\r\n\r\nPlease do not repost position.<\/pre>\n","protected":false},"author":43,"featured_media":100040417,"template":"","meta":{"_monsterinsights_skip_tracking":false,"_promoted":"","_job_location":"","_application":"http:\/\/jobs.gem.com\/stackyak\/am9icG9zdDrauilGE5SD2AfJ04PacNuT","_company_name":"StackYak","_company_website":"https:\/\/stackyak.ai","_company_tagline":"","_company_twitter":"","_company_video":"","_filled":0,"_featured":0,"_remote_position":0,"_job_salary":"","_job_salary_currency":"","_job_salary_unit":""},"job_listing_region":[],"job-categories":[23333],"job-types":[2,3],"class_list":["post-100040416","job_listing","type-job_listing","status-publish","has-post-thumbnail","hentry","job_listing_category-ai","job_listing_type-full-time","job_listing_type-part-time","job-type-full-time","job-type-part-time"],"_links":{"self":[{"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/job-listings\/100040416","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/job-listings"}],"about":[{"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/types\/job_listing"}],"author":[{"embeddable":true,"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/users\/43"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/media\/100040417"}],"wp:attachment":[{"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/media?parent=100040416"}],"wp:term":[{"taxonomy":"job_listing_region","embeddable":true,"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/job_listing_region?post=100040416"},{"taxonomy":"job_listing_category","embeddable":true,"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/job-categories?post=100040416"},{"taxonomy":"job_listing_type","embeddable":true,"href":"https:\/\/www.nbn.org.il\/jobboard\/wp-json\/wp\/v2\/job-types?post=100040416"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}