I am currently working on complex document understanding, using iterative reasoning and a hierarchical caching system specialized for tensors and KV cache. If this sounds interesting, please reach out to towaki@outerport.com.

Until recently, I worked as a Research Scientist at NVIDIA Research on the Hyperscale Graphics Systems team with David Luebke and Morgan McGuire.

I was also concurrently a Computer Science PhD student (on leave) at the University of Toronto, supervised by Sanja Fidler and Alec Jacobson, supported by the Royal Bank of Canada, Borealis AI Fellowship.

My interests broadly span machine learning, systems, computer graphics, and computer vision. In particular, I'm interested in compression, retrieval, distribution formats, and infrastructure for multimedia. In the past, I have worked on neural fields [1] and their combination with differentiable data structures [2,3,4] as a new compressed data format for immersive media (3D JPEG!)- as well as foundation models [5] for generating them.

Experience

2024—Present

Co-Founder / CEO Outerport

Chain-of-memory reasoning for complex document understanding.

2020-2023

Research Scientist NVIDIA

Manager: David Luebke, Morgan McGuire

New Experiences Research Group (Hyperscale Graphics Systems).

Publications

A LoRA is Worth a Thousand Pictures

arXiv 2024

A LoRA is Worth a Thousand Pictures

Chenxi Liu, Towaki Takikawa, Alec Jacobson

LoRAs > embeddings for retrieval and clustering.

Lagrangian Hashing for Compressed Neural Field Representations

ECCV 2024

Lagrangian Hashing for Compressed Neural Field Representations

Shrisudhan Govindarajan, Zeno Sambugaro, Ahan Shabhanov, Towaki Takikawa, Weiwei Sun, Daniel Rebain, Nicola Conci, Kwang Moo Yi, Andrea Tagliasacchi

Points + grids + hash tables for neural field compression.

What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

CVPR 2024

What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

Alex Trevithick, Matthew Chan, Towaki Takikawa, Umar Iqbal, Shalini De Mello, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano

Adaptive sampling in rendering = higher fidelity geometry.

Compact Neural Graphics Primitives with Learned Hash Probing

SIGGRAPH Asia 2023

Compact Neural Graphics Primitives with Learned Hash Probing

Towaki Takikawa, Thomas Müller, Merlin Nimier-David, Alex Evans, Sanja Fidler, Alec Jacobson, Alexander Keller

Learned hash probing + hash tables = more compression & faster.

ATT3D: Amortized Text-To-3D Object Synthesis

ICCV 2023

ATT3D: Amortized Text-To-3D Object Synthesis

Jonathan Lorraine, Kevin Xie, Xiaohui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming-Yu Liu, Sanja Fidler, James Lucas

Optimize 3D generation over more prompts -> faster convergence.

Magic3D: High-Resolution Text-to-3D Content Creation

CVPR 2023

⭐ Highlight (Top 2%)

Magic3D: High-Resolution Text-to-3D Content Creation

Chen-Hsuan Lin*, Jun Gao*, Luming Tang*, Towaki Takikawa*, Xiaohui Zeng*, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, Tsung-Yi Lin

Scaling up 3D generation with smarter data structures.

Variable Bitrate Neural Fields

SIGGRAPH North America 2022

Variable Bitrate Neural Fields

Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Müller, Morgan McGuire, Alec Jacobson, Sanja Fidler

Learned indexing = more compressive than hash tables and grids.

A Dataset and Explorer for 3D Signed Distance Functions

JCGT 2022

A Dataset and Explorer for 3D Signed Distance Functions

Towaki Takikawa, Andrew Glassner, Morgan McGuire

Dataset of shapes made with math equations.

Neural Fields in Visual Computing and Beyond

EG / CGF 2022

Neural Fields in Visual Computing and Beyond

Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, Srinath Sridhar

Survey of representing vector fields with neural networks.

RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis

Tech Report 2022

RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis

Jonathan Tremblay, Moustafa Meshry, Alex Evans, Jan Kautz, Alexander Keller, Sameh Khamis, Charles Loop, Nathan Morrical, Thomas Müller, Koki Nagano, Towaki Takikawa, Stan Birchfield

Really big dataset of synthetic multi-view images.

Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes

CVPR 2021

⭐ Oral Presentation (Top 4%)

Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes

Towaki Takikawa*, Joey Litalien*, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, Sanja Fidler

Differentiable data structures + neural fields + GPU programming = fast.

Gated-SCNN: Gated Shape CNNs for Semantic Segmentation

ICCV 2019

Gated-SCNN: Gated Shape CNNs for Semantic Segmentation

Towaki Takikawa*, David Acuna*, Varun Jampani, Sanja Fidler

Pay attention to object boundaries for better computer vision.

Education

2020—(on leave)

University of Toronto

Ph.D. in Computer Science

Advisor: Sanja Fidler and Alec Jacobson

2016—2020

University of Waterloo

B.S. in Computer Science

Portfolio

NVIDIA Kaolin Core

NVIDIA Kaolin Core

GPU / CUDA kernels for 3D machine learning.

NVIDIA Kaolin Wisp

NVIDIA Kaolin Wisp

GPU-accelerated library for neural fields.

WATonomous

WATonomous

Built an autonomous vehicles from scratch.