Does audio buffer size also affect MIDI latency?

If I insert MIDI data in processBlock (as in e.g. an arpeggiator), MIDI will be updated with a frequency dependent on the audio buffer size. The smaller the audio buffer the more frequent MIDI data is inserted.
So, do smaller audio buffers result in smaller MIDI latency?
Does setting the MIDI timestamp for inserted MIDI messages really matter?