We present the first results of a fast neutron tracker based on a silicon voxel detector combined with a plastic scintillators. The concept of the voxel detector is based on a multilayer structure composed of stacked Timepix devices forming a 3D matrix of sensitive elements – voxels. Such voxel detector maintains compact size while having greatly improved detection efficiency and tracking capabilities. Such voxel detector can be used in various fields (e.g. Compton camera). In this contribution we present application of the voxel detector for tracking of fast neutrons. The multilayer structure of the voxel detector is interlaced with thin layers of plastic scintillators with attached silicon photo multipliers (SiPM). Fast neutron recoils proton from the plastic scintillator. The recoiled proton loses part of its energy in the scintillator and hits the Timepix device. It was already proved that it is possible, from the data measured by the SiPM and Timepix, determine the energy, position and direction of the recoiled proton allowing reconstruction of either neutron energy or its direction. If the scattered neutron undergoes the second scattering in another layer of the voxel detector then it is possible to determine both: the neutron energy and direction. This paper shows experimental results obtained with a prototype of the voxel detector compared with Monte-Carlo simulations. The work has been carried out within the Medipix2 collaboration.