Hi, I'm Xun Tu.
A
I am a Ph.D. student specializing in robotics and computer vision. My research focuses on developing intelligent systems on the quadrupled robotic platform SPOT, to enable it to perceive, approach and manipulate the
target object in mobile manipulation tasks, in an accurate, stable, and efficient way.
Feel free to check out my CV and get in touch!
About
Currently, I am a Computer Science PhD Student at University of Minnesota, Twin Cities. I enjoy solving complex mathematics and programming problems, and deploying the algorithms on real-world robotic platforms. I received my bachelor's from both University of Michigan, Ann Arbor and Shanghai Jiao Tong University through a dual-degree program. I have later received my master's degree in University of Michigan. Those studies have enhanced my understanding in the technologies involving neural networks, Bayesian filters, model-based system control, generative models, etc. I am interested in solving perception & manipulation problems for robots, such as few-shot policy learning & generalization, object-centric representation learning, mobile manipulation, etc.
- Languages: Python, C++, JavaScript, HTML/CSS, C, Bash,Java
- Libraries: NumPy, OpenCV, Diffuser, Scipy
- Frameworks: PyTorch, ROS, ROS2, TorchRL
- Simulation Environment: Isaac-Lab, Genesis
- Mathematics Platform: Matlab, Mathematica
- Tools & Technologies: Git, Docker
Looking for an internship opportunity in the field of Robotics, Computer Vision, Control, Software Development, etc. Open for the position as a research assistant or software program developer. Eager to combine my academic experiences and contribute to a professional project in the field, where I can acquire professional development experiences and personal growth.
Experience
Research Assistant
- Developed and managed the control programs to deploy on the mobile manipulation platform SPOT in the lab to enable it to perceive the surroundings, extract out essential information, and execute manipulation tasks
- Build up a pipeline to detect the the obstacle in the navigation path and remove it automatically using SPOT's arm and gripper, improving the robot's autonomy in mobile manipulation tasks.
- Bring up and Deploy an algorithm on SPOT to enable it to grasp the target large object in any arbitrary shape during a mobile manipulation task. The paper can be found at: IEEE link
- Currently, developing an algorithm to enable SPOT to learn a few-shot mobile manipulation policy to move the target object after a successful grasp, using diffusion policy and Reinforcement Learning.
- Tools: Python, Pytorch, TorchRL, ROS/ROS2, Docker, Diffuser
- Helped in organizing and teaching the course sessions for CSCI 5551: Introduction to Intelligent Robotic Systems and CSCI 5561: Computer Vision
- Devised a customized project to enhance students' understanding of real-world robotic systems using turtlebot3
- Devised a series of Jupyter notebooks to help students understand the fundamental concepts of computer vision, including image processing, feature detection, object recognition, etc.
- Helped to create and grade assignments, projects, and exams for both courses
- Tools: Python, ROS, Jupyter, NumPy, PyTorch
- Help in collecting the data establishing a sample dataset with images in various weather conditions for people to test their SLAM algorithms;
- Test the calibration methods and write technical documentations;
- Test several existing SLAM algorithms (e.g. ORB-SLAM2) and evaluate the performance on the collected data;
- Tools: Python, C++
- Help in grading the assignments in the class EECS 478: Logic Circuit Synthesis and Optimization;
- Tools: C++
Projects
Generalize the few-shot mobile manipulation policy
Grasp the target large object in any arbitrary shape
- Keywords: Grasping, Model Decomposition, Robotics
- Developed to grasp the large objects, such as a chair, that are common in mobile manipulation tasks, but rare in traditional tabletop scenarios.
- Pipeline:
- Decompose the target object into primitives at first
- Generate grasp poses on individual primitives as candidates
- Rank the grasp pose candidates & Filter out the invalid ones
- Validated on the real-world robotic platform
Remove the obstacle on the navigation routine path
- Keywords:Object Detection, Grasping, Motion Planning, ROS
- Use libraries from Boston Dynamics to schedule a path to the target location
- Use OpenCV for image processing
- Use DINOv2, a visual feature extractor to detect the obstacles and determine the part to grasp
- Use libraries from Boston Dynamics to move the obstacle out of the planned path
A dataset to test SLAM algorithms in adverse weather conditions
- Keywords:SLAM, IMU, Sensor Calibration
- Helped in establishing a sample dataset with images in various weather conditions for researchers to test their SLAM algorithms;
- Helped in collecting the visual and IMU data;
- Validated the calibration methods and wrote technical documentations;
- Evaluated several existing SLAM algorithms (e.g. ORB-SLAM2);
A custom robot to go along the assigned waypoints
A algorithm to remove the motion noise in 3D printing process
- Keywords:Feedforward Control, B-spline functions, 3D Printing
- Studied the proposed FBS method in reducing the vibrations in 3D printing for linear systems;
- Proposed to use Newton-Gaussian methods to extend the method to reduce the vibrations for nonlinear systems;
- Wrote up Matlab codes for some starting tests on this method;
A prototype of autonomous excavator that can avoid obstacles while navigating its arm
- Keywords:ROS, Kalman Filter, A* algorithm, Forward/Inverse kinematics
- Developed ideas to avoid the obstacle automatically, while the excavator is operating its arm, based on one excavator prototype provided by BuilderX corporation;
- Helped in collecting point cloud data from a depth camera;
- Developed the motion planning and robotic arm control algorithms;
- Deployed the sensors to read joint angles, including writing the communication programs and filtering out the noises;
A simulated processor for a computer to execute instructions out-of-order [1]
- Keywords:Computer Architecture, Out-of-Order Execution, Verilog
- Helped in developing a P6‐style Out‐of‐Order instruction processor that can work correctly and efficiently to process the provided instructions;
- Designed and implemented the memory I/O components, especially the caches;
- Benchmarked the processor performance;
Skills
Languages
Python
C++
HTML5
CSS3
Shell Scripting
Libraries
NumPy
OpenCV
Diffusers
scikit-learn
matplotlib
Frameworks
PyTorch
ROS/ROS2
TorchRL
Simulation Environment
Genesis
Isaac Sim/Lab
Other
Git
Matlab
Mathematica
Jupyter
Education
University of Minnesota, Twin Cities
Minneapolis, USA
Time: 2023.08 - Present
Degree: Ph.D. in Computer Science
GPA: 3.61/4.0 (Top 20%)
- Advanced Algorithms
- Markov Decision Processes
- Generative Models
- Deep Reinforcement Learning
Relevant Courseworks:
University of Michigan, Ann Arbor
Ann Arbor, USA
Time: 2021.09 - 2023.05
Degree: Master of Science in Electrical & Computer Engineering
GPA: 4.0/4.0 (Top 10%)
- Advanced Computer Vision
- Bayesian Filter & State Estimation
- Robotics
- Linear System Control
Relevant Courseworks:
University of Michigan, Ann Arbor
Ann Arbor, USA
Time: 2019.09 - 2021.05
Degree: Bachelor of Science in Computer Engineering
GPA: 3.74/4.0 (Top 16%)
- Computer Vision
- Embedded Systems
- Computer Architecture
Relevant Courseworks:
Joint Institute of Shanghai Jiao Tong University and University of Michigan
Shanghai, China
Time: 2017.09 - 2021.08
Degree: Bachelor of Science in Electrical & Computer Engineering
GPA: 3.72/4.00 (Top 6%)
- Data Structures and Algorithms
- Probabilities and Statistics
- Linear Algebra
- Linear System Control
- Analog Circuits
- Digital Circuits
Relevant Courseworks:
Contact
Gratitude
I would like to express my gratitude to
- Varad Bhogayata for his website template at http://github.com/varadbhogayata.
- The author of the blog: https://www.slideserve.com/cmcewen/computer-architecture-the-p6-microarchitecture-an-example-of-an-out-of-order-micro-processor-powerpoint-ppt-presentation for the illustration on P6-style instruction processor