I'm trying something like extracting audio features for each video frame. I know there are 30 video frames and 16000 audio frames per second in the video file. I'm using pyAudioAnalysis python lib to achieve the goal but no success.Here is my code.
from __future__ import print_functionfrom pyAudioAnalysis import audioBasicIOfrom pyAudioAnalysis import ShortTermFeatures,MidTermFeaturesimport matplotlib.pyplot as pltimport os,shlex, subprocessimport pandas as pdimport numpy as np# -acodec pcm_s16le -vn -ar 16000command_line = "ffmpeg -i test.mp4 -ac 1 -ar 16000 -vn test_mono.wav"#command_line = "ffmpeg -i test.mp4 -ab 160k -ac 2 -ar 44100 -vn test_stereo.wav"args = shlex.split(command_line)print(args)processResult = subprocess.call(args) # Success!print(processResult)[SamplingRate, signals] = audioBasicIO.read_audio_file("test_mono.wav")print(SamplingRate)Output
['ffmpeg', '-i', 'test.mp4', '-ac', '1', '-ar', '16000', '-vn', 'test_mono.wav']1
16000
#mid_feature_extraction(signal, sampling_rate, mid_window, mid_step,short_window, short_step):MidFeatures,ShortFeatures,MidFeatureLabels=MidTermFeatures.mid_feature_extraction(signals, SamplingRate, 0.043*SamplingRate,0.043*SamplingRate,0.00016*SamplingRate, 0.00016*SamplingRate)print('Mid Features Extr Success')MidFeatures_Dataframe = pd.DataFrame(data=MidFeatures.transpose(), columns=MidFeatureLabels)print(type(MidFeatures))#print(MidFeatures)print(type(ShortFeatures))#print(ShortFeatures)print(type(MidFeatureLabels))#print(MidFeatureLabels)MidFeatures_Dataframe.to_csv('Audio_MidFeatures.csv')print('Mid Features File Success')Output
Mid Features Extr Success
Mid Features File Success
As per my calculations I should get 338 rows of audio features, and after a long time of struggle I'm getting 326 with the above parameters but still don't know how. If anyone can help me how, window and steps are working here. I know the basic concepts of window and step as work in CNN but not getting in this context.