This paper presents a cuckoo search algorithm to minimize makespan for a semiconductor final testing scheduling problem. Each solution is a two-part vector consisting of a machine assignment and an operation sequence. In each iteration, a parameter feedback control scheme based on reinforcement learning is proposed to balance the diversification and intensification of population, and a surrogate model is employed to reduce computational cost. According to the Rechenberg's 1/5 Criterion, reinforcement learning uses the proportion of beneficial mutation as feedback. As a result, the surrogate modeling only needs to evaluate the relative ranking of solutions. A heuristic approach based on the smallest position value rule and a modular function is proposed to convert continuous solutions obtained from Levy flight into discrete ones. The computational complexity analysis is presented, and various simulation experiments are performed to validate the effectiveness of the proposed algorithm.