We aimed to assess the impact of smoking on impaired insulin secretion (IIS) in a Japanese population. This study included community residents who underwent annual multiphasic health screening examinations, including a standard 75-g oral glucose tolerance test. The cohort included 768 men aged 30-69 years who had six eligibility criteria at baseline examination between April 2006 and March 2007: (1) a fasting plasma glucose < 7.0 mmol/L; (2) a 2-h postload glucose < 11.1 mmol/L; (3) a HbA1c (NGSP) < 6.5%; (4) a insulinogenic index (ΔI/ΔG) > 51.7 pmol insulin/mmol glucose; (5) a HOMA-IR < 2.5; and (6) no history of diabetes. ΔI/ΔG was calculated using the difference in the values of 30-min and fasting serum insulin divided by the difference in the values of 30-min and fasting plasma glucose value. We used the Japan Diabetes Society criteria to define IIS. ΔI/ΔG ≤ 51.7 was indicative of IIS. We followed up the 768 men annually until March 2011. Cox proportional hazards regression was used to estimate hazard ratios (HRs) and 95% confidence intervals (CIs). We adjusted for age, family history of diabetes, alcohol consumption, BMI, systolic blood pressure, log-transformed triglycerides, log-transformed γ-glutamyltransferase, and log-transformed high-sensitive CRP. At baseline, 144 men were current smokers who were smoking ≥ 20 pack-years, 46 men were current smokers who were smoking < 20 pack-years, 350 men were past smokers, and 228 men were non-smokers. The mean follow-up was 2.5 years (total person-years: 1957) and 354 men developed IIS during that period. The multivariable-adjusted HRs for IIS were 1.65 (95% CI: 1.22-2.24) in current smokers who were smoking ≥ 20 pack-years, 1.44 (95% CI: 0.87-2.38) in current smokers who were smoking < 20 pack-years, and 1.09 (95% CI: 0.84-1.41) in past smokers compared with non-smokers. In conclusion, smoking was a risk factor for IIS.