trying to make a wav/sunvox visualizer: a pixilang learner's thread

Pixilang programming language
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

hello, i'm trying to make a visualizer for my music that will use a wav file for audio but also get data from a sunvox file. i'm new to pixilang, and i will have a lot of questions on how to do things with it, so instead of clogging this board with threads i'll just put them all in this single thread

my questions will start below
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

first question: how can i get the demo lib to automatically stop when the end of a wav file is reached? i thought that the demo_sound_len variable in the demo lib file was what i'm looking for, but it isn't. also, how do i even tell the main boot.pixi file to stop at that point?

thank you!
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

AutumnCheney wrote: Fri Jun 24, 2022 5:08 am i thought that the demo_sound_len variable in the demo lib file was what i'm looking for, but it isn't. also, how do i even tell the main boot.pixi file to stop at that point?
looks like i was right the first time, that was what i was looking for

i had to divide it by the demo_sound_rate variable (which is the sample rate of the file), then multiply it by 1000 to convert it to milliseconds

then, i had to pass that to the length of a DEMO_STOP scene

here's my code so far, i made a custom init function alongside the default one to do the length calculation and pass it to the scene length. i also configured it to print the length to the screen so i can check it

Code: Select all

include "demo.pixi"

demo_show_fps = 0
//demo_opengl = 1
demo_xsize = 1920
demo_ysize = 1080
//demo_video_export = "video.avi"

demo_load_wav( "media/song.wav" )

demo_init()
custom_init()

demo_add_scene( 0, 0, 0, scene1 )
demo_add_scene( 0, 0, wav_length, DEMO_STOP )

demo_play()
demo_deinit()

fn scene1( $t1, $t2, $len )
{
    //$t1 - current time;
    //$t2 - current time relative to the beginning of the scene;
    //$len - scene length;
    //time function = 0 (main); time unit = millisecond;
    clear( #203040 )


    print( wav_length_str , 0 , 0)
    

}

fn custom_init()
{
	wav_length = demo_sound_len / demo_sound_rate * 1000
	wav_length_str = ""
	num_to_str( wav_length_str, wav_length )
}

show_memory_debug_messages( 1 )
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

now, next question: i'd like to get started with displaying a spectrum (which i asked about here). where could i get started with that, and how could i apply the fft() function to individual audio frames?
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
NightRadio
Site Admin
Posts: 3941
Joined: Fri Jan 23, 2004 12:28 am
Location: Ekaterinburg. Russia
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by NightRadio »

how can i get the demo lib to automatically stop when the end of a wav file is reached?
Here is the simplest way:

Code: Select all

demo_add_scene( 0, 1, demo_sound_len, DEMO_STOP )
Time function (second parameter) = 1.
Available time functions:
0 - main, in milliseconds;
1 - set automatically, depending on the sound (pcm, sunvox, etc.); in frames (pcm), lines (sunvox), or other sound units;

how could i apply the fft() function to individual audio frames?
First try to get these frames :)
In case of SunVox you can use sv_get_module_scope() https://warmplace.ru/soft/sunvox/sunvox ... ule_scope2
using the demo_sound_sv object.
In case of WAV you can get the waveform from the demo_sound_pcm_stream container:

Code: Select all

demo_add_scene( 0, 1, 0, wav_scene )
fn wav_scene4( $t1, $t2, $len )
{
    //time function = 1 (WAV player); time unit = sample (frame);
    //current frame (left channel) = demo_sound_pcm_stream[ $t1 * 2 ]
    //next frame = demo_sound_pcm_stream[ ( $t1 + 1 ) * 2 ]
    //etc
}
If you want to not only receive frames, but also change them (filter) - modify demo_sound_fn_sunvoxplayer() or demo_sound_fn_pcmplayer() in the demo.pixi
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

NightRadio wrote: Sat Jun 25, 2022 11:01 am Here is the simplest way:

Code: Select all

demo_add_scene( 0, 1, demo_sound_len, DEMO_STOP )
Time function (second parameter) = 1.
Available time functions:
0 - main, in milliseconds;
1 - set automatically, depending on the sound (pcm, sunvox, etc.); in frames (pcm), lines (sunvox), or other sound units;
oh cool! this seems simpler than manually calculating the wav length and passing that to the scene length

and someone on discord helped me apply an fft to the pcm sound frames, so that's taken care of

here's the boot.pixi code now

Code: Select all

include "demo.pixi"

demo_show_fps = 0
//demo_opengl =1
demo_xsize = 1920
demo_ysize = 1080
//demo_video_export = "video.avi"

fft_size = 512
fft_im = new( fft_size, 1, FLOAT )
fft_re = new( fft_size, 1, FLOAT )

demo_load_wav( "media/song.wav" )

demo_init()

demo_add_scene( 0, 0, 0, scene1 )
demo_add_scene( 0, 1, demo_sound_len, DEMO_STOP )

demo_play()
demo_deinit()

fn scene1( $t1, $t2, $len )
{
    //$t1 - current time;
    //$t2 - current time relative to the beginning of the scene;
    //$len - scene length;
    //time function = 0 (main); time unit = millisecond;
    clear( #203040 )

    audio_fft()
    

}

fn audio_fft()
{
    copy(fft_re, demo_sound_pcm_stream, 0, demo_sound_pcm_stream_ptr * demo_sound_channels, fft_size, 1, demo_sound_channels)
    clean(fft_im)
    fft(0, fft_im, fft_re)

    i = 0 while i < fft_size
    {
        im = fft_im[i]
        re = fft_re[i]
        amplitude = sqrt(im * im + re * re) / fft_size

        x = i - fft_size/2
        y = -amplitude * 300

        line(x, y, x, 0, get_color(255, 255, 255))

        i + 1
    }
}

show_memory_debug_messages( 1 )
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

now, i'd like to display the spectrum logarithmically, like in sunvox's spectrum analyser

any pointers or hints on how to do this? thank you!

btw correct me if i'm wrong but this shouldn't involve an lft, right? it's just a display thing as opposed to a algorithm change?
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

AutumnCheney wrote: Sun Jun 26, 2022 6:49 am now, i'd like to display the spectrum logarithmically, like in sunvox's spectrum analyser

any pointers or hints on how to do this? thank you!

btw correct me if i'm wrong but this shouldn't involve an lft, right? it's just a display thing as opposed to a algorithm change?
bump because i became interested in doing this again
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

looking at fft2.pixi, i see there's a commented out section that gives some code to apply some transformations to analyze the spectrum

my question is, in that code, what is "k"? is that some kind of loop index? and how would i use that code in my code?
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
NightRadio
Site Admin
Posts: 3941
Joined: Fri Jan 23, 2004 12:28 am
Location: Ekaterinburg. Russia
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by NightRadio »

what is "k"?
It is the same as "i" :)
ainegil
Posts: 168
Joined: Thu Sep 22, 2022 11:37 pm

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by ainegil »

Heres a basic version for reference

Code: Select all

include "../../lib/mjpeg.pixi"
fps= 25
filename = "sound_files/Echos Flower.wav"
wav = load( filename )

wav_ptr = 0
wav_size = get_size( wav ) //number of frames
wav_channels = wav.channels
wav_amp_max = 256
type = get_type( wav )
if type == INT16 { wav_amp_max = 1 << 15 }
if type == INT32 { wav_amp_max = 1 << 30 }
if type == FLOAT32 { wav_amp_max = 1 }

ftsize = 1024
bufre = new(ftsize, 1, FLOAT)
bufim = new(ftsize, 1, FLOAT)


set_audio_callback( audio_callback, 0, wav.sample_rate, get_type( wav ), wav_channels, AUDIO_FLAG_INTERP2 )

scr = get_screen()
xsize= get_xsize(scr)



while 1
{
clear()
clean(bufre)
clean(bufim)

$p = wav_ptr
$t = $p / wav_size

$i = 0 
while $i < ftsize{
$v = wav[ $p + $i * wav_channels ] / wav_amp_max
bufre[$i]=$v
$i = $i +1
}

fft(0, bufim, bufre,ftsize)

$i = 0 while $i < ftsize / 2 {

$vy = bufre[$i]*bufre[$i] + bufim[$i]*bufim[$i]
$vy = sqrt($vy)
$vy = 20*log10($vy)
$xx =log2( ($i+1)/ ftsize)*25
line ($xx, 0,$xx,-$vy,WHITE)

$i = $i +1
}

    
    
line (-xsize, 0,xsize,0,WHITE)
	frame( 1000 / fps )
    
    if wav_ptr >= wav_size { breakall }
    while( get_event() ) { if EVT[ EVT_TYPE ] == EVT_QUIT { breakall } }
}


fn audio_callback(
    $stream, 
    $userdata, 
    $channels, 
    $frames, 
    $output_time_in_system_ticks, 
    $in_channels, 
    $latency_in_frames )
{
    if wav_ptr >= wav_size { ret( 0 ) }
    $c = 0 while( $c < wav_channels )
    {
	copy( $channels[ $c ], wav, 0, wav_ptr + $c, $frames, 1, wav_channels )
	$c + 1
    }
    wav_ptr + $frames * wav_channels
    ret( 1 )
}
The scale is log2 so its octave wise
ainegil
Posts: 168
Joined: Thu Sep 22, 2022 11:37 pm

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by ainegil »

Forgot to window so it should be

$win =sin(M_PI*$i /ftsize)
bufre[$i]=$v * $win

For instance
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

ainegil wrote: Fri Oct 07, 2022 12:00 am Heres a basic version for reference

Code: Select all

include "../../lib/mjpeg.pixi"
fps= 25
filename = "sound_files/Echos Flower.wav"
wav = load( filename )

wav_ptr = 0
wav_size = get_size( wav ) //number of frames
wav_channels = wav.channels
wav_amp_max = 256
type = get_type( wav )
if type == INT16 { wav_amp_max = 1 << 15 }
if type == INT32 { wav_amp_max = 1 << 30 }
if type == FLOAT32 { wav_amp_max = 1 }

ftsize = 1024
bufre = new(ftsize, 1, FLOAT)
bufim = new(ftsize, 1, FLOAT)


set_audio_callback( audio_callback, 0, wav.sample_rate, get_type( wav ), wav_channels, AUDIO_FLAG_INTERP2 )

scr = get_screen()
xsize= get_xsize(scr)



while 1
{
clear()
clean(bufre)
clean(bufim)

$p = wav_ptr
$t = $p / wav_size

$i = 0 
while $i < ftsize{
$v = wav[ $p + $i * wav_channels ] / wav_amp_max
bufre[$i]=$v
$i = $i +1
}

fft(0, bufim, bufre,ftsize)

$i = 0 while $i < ftsize / 2 {

$vy = bufre[$i]*bufre[$i] + bufim[$i]*bufim[$i]
$vy = sqrt($vy)
$vy = 20*log10($vy)
$xx =log2( ($i+1)/ ftsize)*25
line ($xx, 0,$xx,-$vy,WHITE)

$i = $i +1
}

    
    
line (-xsize, 0,xsize,0,WHITE)
	frame( 1000 / fps )
    
    if wav_ptr >= wav_size { breakall }
    while( get_event() ) { if EVT[ EVT_TYPE ] == EVT_QUIT { breakall } }
}


fn audio_callback(
    $stream, 
    $userdata, 
    $channels, 
    $frames, 
    $output_time_in_system_ticks, 
    $in_channels, 
    $latency_in_frames )
{
    if wav_ptr >= wav_size { ret( 0 ) }
    $c = 0 while( $c < wav_channels )
    {
	copy( $channels[ $c ], wav, 0, wav_ptr + $c, $frames, 1, wav_channels )
	$c + 1
    }
    wav_ptr + $frames * wav_channels
    ret( 1 )
}
The scale is log2 so its octave wise
thank you so much for the example! now my spectrum is displayed logarithmic like i want it to
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

one last question: how can i get the amplitude of the wav sample from the demo.pixi? i'd like to implement something that flashes with the loudness of the audio file
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
User avatar
AutumnCheney
Posts: 503
Joined: Sun Dec 29, 2019 8:16 am
Location: tahlequah, ok, usa
Contact:

Re: trying to make a wav/sunvox visualizer: a pixilang learner's thread

Post by AutumnCheney »

AutumnCheney wrote: Sat Oct 08, 2022 3:22 am one last question: how can i get the amplitude of the wav sample from the demo.pixi? i'd like to implement something that flashes with the loudness of the audio file
:fool: realized it just now, i can use the demo_sound_pcm_stream container
my website: https://acheney.xyz

it features my music, sunvox content, and social media links!
Post Reply