RESEARCH
My current research interest is in 3D computer vision. We live in a 3D world, and I am thrilled to understand it, and to recreate it. Previously I also worked on quantum information technologies. I serve as a reviewer for Journal TNNLS.
Zero123++: A Single Image to Consistent Multi-view Diffusion Base Model
Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, Hao Su
Careful designs on Stable Diffusion for high-quality consistent multi-view generation from single image input.
OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding
Minghua Liu*, Ruoxi Shi*, Kaiming Kuang*, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, Hao Su
NIPS'23
3D native VLM. 85.3% zero-shot accuracy on ModelNet40 matching some fully-supervised baselines. 77% top-5 accuracy on 1136-class Objaverse-LVIS. The point cloud encoder can also be used for captioning, point cloud-conditioned image generation and multi-modal retrieval. It is a great step towards open-world understanding of 3D shapes.
Toward Learning Geometric Eigen-Lengths Crucial for Fitting Tasks
Yijia Weng, Kaichun Mo, Ruoxi Shi, Yanchao Yang, Leonidas Guibas
ICML'23
Humans can find compact and meaningful geometric concepts (width, radius, volume etc.) as summaries for objects. In this work we attempt to arm machines with the same capability of conceptual emergence and explore how this may help with robotic tasks.
RendNet: Unified 2D/3D Recognizer with Latent Space Rendering
Ruoxi Shi, Xinyang Jiang, Caihua Shan, Yansen Wang, Dongsheng Li
CVPR'22 (Oral)
VG (Vector Graphics) have been ubiquitous in our daily life with vast applications in engineering, architecture, designs, etc. We connect VG and RG (Raster Graphics) with a latent space rendering technique to get the best of both worlds: the infinite resolution and high-level topology information in VG, and the availability and natural error filtering in RG.
CPPF: Towards Robust Category-Level 9D Pose Estimation in the Wild
Yang You, Ruoxi Shi, Weiming Wang, Cewu Lu
CVPR'22
By aggregating votes on pairs of local small patches, the deep learning models are more robust and generalizes better. The sim-to-real transfer performance of pose estimation of CPPF beats previous methods by a large margin.
Skeleton Merger: an Unsupervised Aligned Keypoint Detector
Ruoxi Shi, Zhengrong Xue, Yang You, Cewu Lu
CVPR'21 (Oral)
Detecting aligned 3D keypoints is essential under many scenarios such as object tracking, shape retrieval and robotics. However, it is generally hard to prepare a high-quality dataset for all types of objects due to the ambiguity of keypoint itself. Meanwhile, current unsupervised detectors are unable to generate aligned keypoints with good coverage. In this work we design SkeletonMerger, an auto-encoder architecture based on the skeleton representation, to detect aligned keypoints from objects in an unsupervised manner.
PRIN/SPRIN: On Extracting Point-Wise Rotation Invariant Features
Yang You*, Yujing Lou*, Ruoxi Shi, Qi Liu, Yu-Wing Tai, Lizhuang Ma, Weiming Wang, Cewu Lu
TPAMI'21
SO(3) rotation invariance is an interesting property of 3D geometry. We present the networks (S)PRIN (Sparse Point-wise Rotation Invariant Networks) based on SO(3) group convolution. They achieve state-of-the-art results on rotation-invariant representation learning.
Training a Quantum PointNet with Nesterov Accelerated Gradient Estimation by Projection
Ruoxi Shi, Hao Tang, Xian-Min Jin
NIPS QTNML Workshop'21
PointNet implemented with quantum circuits with special designs that provide exponential speed-up on the per-point layers. Quantum PointNet is tested on Ion-Q processors with ModelNet3 accuracy on-par with classical PointNet.
TensorFlow Solver for Quantum PageRank in Large-Scale Networks
Hao Tang*, Ruoxi Shi*, TianShen He*, YanYan Zhu, TianYu Wang, Marcus Lee, XianMin Jin
Science Bulletin'21
Quantum Stochastic Walk (QSW), of which the Quantum PageRank algorithm is based on, is a foundamental formulation of quantum many-body interaction with strong local restriction fields. In this work we reformulate the QSW master equation to utilize more structure of the problem and develop a simulator that fully leverages the power of GPU parallel computation. We achieved 500x lower memory consumption and 2000x speed-up quantum page-ranking a real-world large-scale network of airports.
