Other steps expected to be announced by the Trade Secretary the day after the Budget include stronger takeover legislation to reduce political influence on merger decisions, a shake-up of the Training and Enterprise Councils and changes in bankruptcy law.
There will also be a new consumer strategy to boost competition. Audits of prices charged by car dealers, supermarkets and high-street retailers will be published in the summer.
The small business bureau will borrow ideas from America's Small Business Administration. The Government is aware that smaller firms are worried about the amount of new rules and tax changes they have to implement, from the minimum wage and working time directive to the Working Families Tax Credit.
The new body will help companies with up to 20 employees comply with changes imposed on them by government. It is also intended to act as an advocate for small firms within government, reflecting the DTI's concern that they have not always been heard in the past.
The concentration of government resources for small business into one agency should also help to ensure that they are used more efficiently.
Mr Byers will also make a series of announcements following on from the DTI's Competitiveness White Paper and the Budget's emphasis on encouraging enterprise. The aim will be to strengthen the competitiveness of the British economy, as well as overcome the alarm business organisations have begun to express about the burden of new red tape.
The moves will be welcomed by employers' organisations. Some of these, including the British Chambers of Commerce, have been advocating a small business administration for the UK for some time.
Yesterday Adair Turner, the director-general of the Confederation of British Industry, urged targeted assistance for smaller companies.
Mr Turner said the Budget should include measures that would boost their investment and spending on research and development.
In a television interview yesterday he said: "We would obviously be concerned if there were any increase in the burdens on business after a significant increase in some of the tax burdens and also administrative burdens over the last few years."