Pybullet is an open source simulation platform created by Facebook designed to train physical agents (like robots) in a 3D environment. It provides a physics engine for all solid and soft bodies. It is widely used in robotics, artificial intelligence, and game development.
Robotic arms they are popular for their speed, accuracy, and ability to operate in dangerous environments. They are used for tasks such as heating, assembly and material handling, especially in industrial settings (such as manufacturing).
There are two ways for a robot to do a job:
- Manual Control – It requires a person to plan clearly every action. It is best suited for planned operations, but is complicated by uncertainty and requires dynamic parameter tuning for every new situation.
- Artificial intelligence – Allows the robot to learn the best actions through trial and error to achieve a goal. Therefore, it can adapt to changing situations by learning rewards and punishments without a defined plan (reinforcement learning).
In this article, I will show you how create a 3D environment with Pybullet Manual control of a robotic arm. I will present some useful Python code that can be easily used in other similar cases (just copy, paste, run) and go) go through every line of code to repeat and repeat and repeat and repeat and repeat and repeat and repeat this example.
Timing
Pybullet can be easily installed with one of the following commands (if peach failed, listen they should definitely work):
pip install pybullet
conda install -c conda-forge pybulletPybullet comes with a set of settings Urdf files (Integrated Robot Format), which are XML Schemas that describe the structure of objects in 3D space (ie cube, sphere, robot).
import pybullet as p
import pybullet_data
import time
## setup
p.connect(p.GUI)
p.resetSimulation()
p.setGravity(gravX=0, gravY=0, gravZ=-9.8)
p.setAdditionalSearchPath(path=pybullet_data.getDataPath())
## load URDF
plane = p.loadURDF("plane.urdf")
cube = p.loadURDF("cube.urdf", basePosition=[0,0,0.1], globalScaling=0.5, useFixedBase=True)
obj = p.loadURDF("cube_small.urdf", basePosition=[1,1,0.1], globalScaling=1.5)
p.changeVisualShape(objectUniqueId=obj, linkIndex=-1, rgbaColor=[1,0,0,1]) #red
while p.isConnected():
p.setRealTimeSimulation(True)Let's go through the code above:
- When you can connect to the physics engine, you need to specify if you want to open the Visual display (
p.GUI) or not (p.DIRECT). - Cartesian space has them 3 dimensions: x-axis (forward / back / back), y-axis (left / right), iz-axis (up / down).
- It is a common practice to set up weight at
(x=0, y=0, z=-9.8)It simulates the gravity of the earth, but one can change it based on the purpose of the simulation. - Usually, you need to Import the plane To build the ground, otherwise things just fall forever. If you want something set down, then set it
useFixedBase=True(of course – It's a lie by mistake). I import things withbasePosition=[0,0,0.1]which means they are 10cm above the ground. - Revelation can be interpreted internally real time and
p.setRealTimeSimulation(True)or faster (CPU-time) withp.stepSimulation(). Another way to set the real time is:
import time
for _ in range(240): #240 timestep commonly used in videogame development
p.stepSimulation() #add a physics step (CPU speed = 0.1 second)
time.sleep(1/240) #slow down to real-time (240 steps × 1/240 second sleep = 1 second)A robot
Currently, our 3D environment is made of a small object (small red cube), and a table (large cube) arranged on the ground (plane). I will add a robotic arm For the task of picking up a small object and placing it on the table.
Pybullet It has an automatic robotic arm configured on the back of the fanra panda robot.
robo = p.loadURDF("franka_panda/panda.urdf",
basePosition=[1,0,0.1], useFixedBase=True)
The first thing to do is analyze i Others (hard parts) and they are members (connection between two solid parts of the robot. For this purpose, you can just use p.DIRECT Since there is no need to consult.
import pybullet as p
import pybullet_data
## setup
p.connect(p.DIRECT)
p.resetSimulation()
p.setGravity(gravX=0, gravY=0, gravZ=-9.8)
p.setAdditionalSearchPath(path=pybullet_data.getDataPath())
## load URDF
robo = p.loadURDF("franka_panda/panda.urdf",
basePosition=[1,0,0.1], useFixedBase=True)
## joints
dic_info = {
0:"joint Index", #starts at 0
1:"joint Name",
2:"joint Type", #0=revolute (rotational), 1=prismatic (sliding), 4=fixed
3:"state vectorIndex",
4:"velocity vectorIndex",
5:"flags", #nvm always 0
6:"joint Damping",
7:"joint Friction", #coefficient
8:"joint lowerLimit", #min angle
9:"joint upperLimit", #max angle
10:"joint maxForce", #max force allowed
11:"joint maxVelocity", #max speed
12:"link Name", #child link connected to this joint
13:"joint Axis",
14:"parent FramePos", #position
15:"parent FrameOrn", #orientation
16:"parent Index" #−1 = base
}
for i in range(p.getNumJoints(bodyUniqueId=robo)):
joint_info = p.getJointInfo(bodyUniqueId=robo, jointIndex=i)
print(f"--- JOINT {i} ---")
print({dic_info[k]:joint_info[k] for k in dic_info.keys()})
## links
for i in range(p.getNumJoints(robo)):
link_name = p.getJointInfo(robo, i)[12].decode('utf-8') #field 12="link Name"
dyn = p.getDynamicsInfo(robo, i)
pos, orn, *_ = p.getLinkState(robo, i)
dic_info = {"Mass":dyn[0], "Friction":dyn[1], "Position":pos, "Orientation":orn}
print(f"--- LINK {i}: {link_name} ---")
print(dic_info)
All robots have “background“, the part of its body that connects everything (like our skin spine). Looking at the code output above, we know that the robot arm has 12 links. They are connected like this:
Movement control
To make the robot do something, you have to move its joints. There are 3 main types of control, in all applications of Newton's Laws:
- Position control: Basically, you tell the robot “Go here”. Technically, you set a Target positionthen apply Force gradually to move the joint to its current position towards the target. As it approaches, the applied forces decrease to avoid extremes and eventually balance the resistance forces that resist this (ie friction, gravity) holding the joint in place.
- Volicity to manage: Basically, you tell the robot “Move at this speed”. Technically, you set a target velocityand apply the force gradually to bring the velocity from zero to the target, then maintain it by balancing the applied force with the force of resistance (meaning drag).
- Force / Torque to manage: Basically, 'you push the robot and see what happens “. Technically, you directly put the force applied to the joint, and the resulting movement depends on the reduction of weight, inertia, and nature. As a side note, in physics, the term “pressure“Used for vertical movement (push/pull), while”promise“It shows a circular movement (twisting/turning).
We can use p.setJointMotorControl2() One integrated move, too p.setJointMotorControlArray() applying force to multiple joints simultaneously. For example, I will do a position control position by assigning a random target to each case.
## setup
p.connect(p.GUI)
p.resetSimulation()
p.setGravity(gravX=0, gravY=0, gravZ=-9.8)
p.setAdditionalSearchPath(path=pybullet_data.getDataPath())
## load URDF
plane = p.loadURDF("plane.urdf")
cube = p.loadURDF("cube.urdf", basePosition=[0,0,0.1], globalScaling=0.5, useFixedBase=True)
robo = p.loadURDF("franka_panda/panda.urdf", basePosition=[1,0,0.1], useFixedBase=True)
obj = p.loadURDF("cube_small.urdf", basePosition=[1,1,0.1], globalScaling=1.5)
p.changeVisualShape(objectUniqueId=obj, linkIndex=-1, rgbaColor=[1,0,0,1]) #red
## move arm
joints = [0,1,2,3,4,5,6]
target_positions = [1,1,1,1,1,1,1] #<--random numbers
p.setJointMotorControlArray(bodyUniqueId=robo, jointIndices=joints,
controlMode=p.POSITION_CONTROL,
targetPositions=target_positions,
forces=[50]*len(joints))
for _ in range(240*3):
p.stepSimulation()
time.sleep(1/240)
The real question is, “What is the correct position for each combination?“The answer is inverse kinematics, the mathematical process of calculating the parameters needed to position the robot in a given position and each position defining the angle it intersects with the hand. Therefore, we will ask Pybullet to find target positions in Carresian space with p.calculateInverseKinematics().
obj_position, _ = p.getBasePositionAndOrientation(obj)
obj_position = list(obj_position)
target_positions = p.calculateInverseKinematics(
bodyUniqueId=robo,
endEffectorLinkIndex=11, #grasp-target link
targetPosition=[obj_position[0], obj_position[1], obj_position[2]+0.25], #25cm above object
targetOrientation=p.getQuaternionFromEuler([0,-3.14,0]) #[roll,pitch,yaw]=[0,-π,0] -> hand pointing down
)
arm_joints = [0,1,2,3,4,5,6]
p.setJointMotorControlArray(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndices=arm_joints,
targetPositions=target_positions[:len(arm_joints)],
forces=[50]*len(arm_joints))
Please let me know what I used p.getQuaternionFromEuler() Converting 3D angles (easy for people to understand) into 4D, by “quaterny“It's easy for physics engines to be paid. If you want to get technology, in quaternion (x, y, z, w)The first three elements define the axis of rotation, and the 4th dimension includes the amount of rotation (a drop/sin).
The code above instructs the robot to move its hand in a certain position over an object using Inverse Kinematics. We catch where the little red cube is sitting on the ground with p.getBasePositionAndOrientation()and we use the information to calculate a target position for members.
Meet things
The robot arm has a hand (“gripper”), so it can be opened and closed to grasp objects. From the previous analysis, we know that “fingers” can go within (0-0.04). Therefore, you can set the target position as a lower limit (Hand is closed) or upper limit (Open by hand).
In addition, I want to make sure that the arm is holding a small red cube while moving around. You can use p.createConstraint() to do a Temporary physics bond Between the Robot Gripper and the object, to match the robot's hand. In the real world, the gripper will apply force by friction and contact with the object to sink the object so it does not fall. But, since Pybullet is a very simple simulator, I'll just take this shortcut.
## close hand
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=9, #finger_joint1
targetPosition=0, #lower limit for finger_joint1
force=force)
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=10, #finger_joint2
targetPosition=0, #lower limit for finger_joint2
force=force)
## hold the object
constraint = p.createConstraint(
parentBodyUniqueId=robo,
parentLinkIndex=11,
childBodyUniqueId=obj,
childLinkIndex=-1,
jointType=p.JOINT_FIXED,
jointAxis=[0,0,0],
parentFramePosition=[0,0,0],
childFramePosition=[0,0,0,1]
)After that, we can move the arm towards the table, using the same strategy as before: Internal Kinematics -> Decorative Position -> Control Actuator.
Finally, when the robot reaches the target position in the Cartesian space, it can open the hand and release the constraints between the object and the arm.
## close hand
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=9, #finger_joint1
targetPosition=0.04, #upper limit for finger_joint1
force=force)
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=10, #finger_joint2
targetPosition=0.04, #upper limit for finger_joint2
force=force)
## drop the obj
p.removeConstraint(constraint)
Full Notebook Control
between Pybulletyou need to provide coverage of all the actions the robot takes. Therefore, it is convenient to have a Utils function that can be accelerated (ie sec =0.1) or reduce it slightly (ie sec = 2) real time (sec = 1).
import pybullet as p
import time
def render(sec=1):
for _ in range(int(240*sec)):
p.stepSimulation()
time.sleep(1/240)Here is the complete code for the simulation we did in this article.
import pybullet_data
## setup
p.connect(p.GUI)
p.resetSimulation()
p.setGravity(gravX=0, gravY=0, gravZ=-9.8)
p.setAdditionalSearchPath(path=pybullet_data.getDataPath())
plane = p.loadURDF("plane.urdf")
cube = p.loadURDF("cube.urdf", basePosition=[0,0,0.1], globalScaling=0.5, useFixedBase=True)
robo = p.loadURDF("franka_panda/panda.urdf", basePosition=[1,0,0.1], globalScaling=1.3, useFixedBase=True)
obj = p.loadURDF("cube_small.urdf", basePosition=[1,1,0.1], globalScaling=1.5)
p.changeVisualShape(objectUniqueId=obj, linkIndex=-1, rgbaColor=[1,0,0,1])
render(0.1)
force = 100
## open hand
print("### open hand ###")
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=9, #finger_joint1
targetPosition=0.04, #upper limit for finger_joint1
force=force)
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=10, #finger_joint2
targetPosition=0.04, #upper limit for finger_joint2
force=force)
render(0.1)
print(" ")
## move arm
print("### move arm ### ")
obj_position, _ = p.getBasePositionAndOrientation(obj)
obj_position = list(obj_position)
target_positions = p.calculateInverseKinematics(
bodyUniqueId=robo,
endEffectorLinkIndex=11, #grasp-target link
targetPosition=[obj_position[0], obj_position[1], obj_position[2]+0.25], #25cm above object
targetOrientation=p.getQuaternionFromEuler([0,-3.14,0]) #[roll,pitch,yaw]=[0,-π,0] -> hand pointing down
)
print("target position:", target_positions)
arm_joints = [0,1,2,3,4,5,6]
p.setJointMotorControlArray(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndices=arm_joints,
targetPositions=target_positions[:len(arm_joints)],
forces=[force/3]*len(arm_joints))
render(0.5)
print(" ")
## close hand
print("### close hand ###")
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=9, #finger_joint1
targetPosition=0, #lower limit for finger_joint1
force=force)
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=10, #finger_joint2
targetPosition=0, #lower limit for finger_joint2
force=force)
render(0.1)
print(" ")
## hold the object
print("### hold the object ###")
constraint = p.createConstraint(
parentBodyUniqueId=robo,
parentLinkIndex=11,
childBodyUniqueId=obj,
childLinkIndex=-1,
jointType=p.JOINT_FIXED,
jointAxis=[0,0,0],
parentFramePosition=[0,0,0],
childFramePosition=[0,0,0,1]
)
render(0.1)
print(" ")
## move towards the table
print("### move towards the table ###")
cube_position, _ = p.getBasePositionAndOrientation(cube)
cube_position = list(cube_position)
target_positions = p.calculateInverseKinematics(
bodyUniqueId=robo,
endEffectorLinkIndex=11, #grasp-target link
targetPosition=[cube_position[0], cube_position[1], cube_position[2]+0.80], #80cm above the table
targetOrientation=p.getQuaternionFromEuler([0,-3.14,0]) #[roll,pitch,yaw]=[0,-π,0] -> hand pointing down
)
print("target position:", target_positions)
arm_joints = [0,1,2,3,4,5,6]
p.setJointMotorControlArray(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndices=arm_joints,
targetPositions=target_positions[:len(arm_joints)],
forces=[force*3]*len(arm_joints))
render()
print(" ")
## open hand and drop the obj
print("### open hand and drop the obj ###")
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=9, #finger_joint1
targetPosition=0.04, #upper limit for finger_joint1
force=force)
p.setJointMotorControl2(bodyUniqueId=robo, controlMode=p.POSITION_CONTROL,
jointIndex=10, #finger_joint2
targetPosition=0.04, #upper limit for finger_joint2
force=force)
p.removeConstraint(constraint)
render()
Lasting
This article has been a lesson in how to be With the power to control the robot arm. We learned about motion control, deflection kinematics, grasping moving objects. A new tutorial on advanced robots is coming.
The complete code for this article: A certain medicine kiki
I hope you enjoyed it! Feel free to contact me with questions and feedback, or just to share your fun projects.
👉 Let's get in touch 👈
(All images are by the author unless otherwise noted)
