In assistive robotics for disabled individuals or telerobotics for remote operations, the human user may need to teleoperate a robot in an unstructured environment. Many tasks involve physical contact between the robot end-effector and environment, e.g., cleaning window, opening door, etc. To successfully execute these tasks, force control is required in the constrained direction together with position control in the complementary direction. In this paper, we consider the problem of human-directed position/force control of robot end-effector interacting with the environment with unknown or uncertain geometry and stiffness. The human user supplies the motion command for the end-effector, while the contact force is autonomously regulated. We assume that the contact force between the robot end-effector and the environment may be measured, but the exact geometric relationship is not available. The measured contact force is used to determine if the robot is in the free space mode or the contact mode. In the free space mode, the user command translates to the end-effector position or velocity. In the contact mode, the user input in the unconstrained direction specifies the motion, while the input in the constrained direction, as determined by the measured contact force, changes the force setpoint. To avoid chattering around the constraint boundary, the transition between the free space mode and contact mode is governed by a hysteresis function with the measured contact force as the input. The proposed approach is implemented on the dual-arm Baxter robot mounted on a mobile base. In the experiment demonstration, a human user commands the mobile manipulator to erase the letters on a white board. For thorough board erasure, sufficient normal force needs to be applied. We show that the user can easily adjust the force setpoint while commanding the end effector motion to successfully execute the task.