Estou tentando encontrar o tamanho de um diretório http/https remoto (não recursivamente) usando um determinado regexp ( \.mp4) na linha de comando, existe uma ferramenta para fazer isso ou preciso analisar o index.html?
EDIT0: Aqui está a saída que eu quero analisar:
$ URL=https://cdn.sermons.love/mp4/Joseph%20Prince/
$ curl -s $URL | html2text -width $COLUMNS
****** Index of /mp4/Joseph Prince ******
[Icon ] Name Last_modified Size
Description
=======================================================================================================================
[[PARENTDIR]] Parent_Directory -
[[ ]] Joseph_Prince_-_Activate_the_Grace_Covenant_Through_Tongues.mp4 2019-04-16 22:32 428M
[[ ]] Joseph_Prince_-_Align_Yourself_With_His_Purpose_and_Prosper.mp4 2019-04-16 22:36 452M
[[ ]] Joseph_Prince_-_Amazing_Things_Happen_When_You_Flow_with_The_Spirit.mp4 2019-04-16 21:48 391M
[[ ]] Joseph_Prince_-_Are_You_Frustrating_The_Favor_of_God.mp4 2019-04-16 21:27 524M
[[ ]] Joseph_Prince_-_As_Jesus_Is,_So_Are_You.mp4 2019-04-16 22:28 894M
[[ ]] Joseph_Prince_-_Blessings_Flow_Through_Grace.mp4 2019-04-30 16:00 761M
[[ ]] Joseph_Prince_-_Break_Every_Bad_Habit_With_Christ.mp4 2019-04-16 22:48 462M
[[ ]] Joseph_Prince_-_Break_Free_from_Addiction_and_Shame.mp4 2019-04-16 22:12 388M
[[ ]] Joseph_Prince_-_Come_As_You_Are_and_Receive_Your_Miracle.mp4 2019-04-16 22:16 444M
[[ ]] Joseph_Prince_-_Discerning_the_Lord's_Body_for_Greater_Health.mp4 2019-04-16 22:38 705M
[[ ]] Joseph_Prince_-_Discover_God's_Way_to_Bless_You.mp4 2019-04-16 21:28 524M
[[ ]] Joseph_Prince_-_Don't_Fight_-_Feed!.mp4 2019-04-16 22:18 453M
[[ ]] Joseph_Prince_-_Draw_The_Blood_Line_Of_Protection.mp4 2019-04-25 13:21 804M
[[ ]] Joseph_Prince_-_Easter_from_New_Creation_Church.mp4 2019-04-16 21:23 843M
[[ ]] Joseph_Prince_-_Enjoy_Jesus'_Supply_And_Delight_His_Heart.mp4 2019-04-16 21:38 609M
[[ ]] Joseph_Prince_-_Experience_God's_Restoration_For_Every_Regret.mp4 2019-04-16 22:37 421M
[[ ]] Joseph_Prince_-_Experience_The_Grace_Revolution.mp4 2019-04-16 21:41 189M
[[ ]] Joseph_Prince_-_Find_Freedom_from_Every_Bondage_and_Addiction.mp4 2019-04-16 22:13 523M
[[ ]] Joseph_Prince_-_Find_True_Fulfillment_In_Life.mp4 2019-04-16 22:20 209M
[[ ]] Joseph_Prince_-_Five_Words_To_Live_By__The_Battle_Is_The_Lord's.mp4 2019-04-16 21:57 226M
[[ ]] Joseph_Prince_-_Fresh_Grace_For_Every_Trial.mp4 2019-04-16 22:14 609M
[[ ]] Joseph_Prince_-_Give_Jesus_Your_Cares_And_Live_Stress-Free.mp4 2019-04-16 21:00 428M
[[ ]] Joseph_Prince_-_God_Is_A_Gracious_Rewarder.mp4 2019-04-16 22:42 224M
[[ ]] Joseph_Prince_-_God's_Blueprint_For_Leadership.mp4 2019-04-16 21:30 776M
[[ ]] Joseph_Prince_-_God's_Perfect_Timing_In_The_Christmas_Story.mp4 2019-04-16 21:29 441M
[[ ]] Joseph_Prince_-_God's_Plan_To_Bless_You.mp4 2019-04-16 21:56 200M
[[ ]] Joseph_Prince_-_Godly_Discipline_And_Correction_Brings_Promotion.mp4 2019-04-16 21:56 776M
[[ ]] Joseph_Prince_-_Got_A_Weakness__God_Can_Use_You!.mp4 2019-04-16 21:25 524M
[[ ]] Joseph_Prince_-_Grace_Leadership_In_Action__How_To_Represent_God's_Heart.mp4 2019-04-16 22:30 732M
[[ ]] Joseph_Prince_-_He_Is_Risen.mp4 2019-04-16 21:47 442M
[[ ]] Joseph_Prince_-_Healing_Flows_When_Grace_Is_Exalted.mp4 2019-04-16 21:51 225M
[[ ]] Joseph_Prince_-_Hear_Jesus_Only_And_Be_Uplifted.mp4 2019-04-16 21:25 436M
[[ ]] Joseph_Prince_-_Hear_the_Preached_Word_and_See_Breakthroughs.mp4 2019-04-16 21:31 459M
[[ ]] Joseph_Prince_-_Hesed_Wisdom_to_Live_Skillfully.mp4 2019-04-16 21:32 389M
[[ ]] Joseph_Prince_-_His_Healing_Is_For_The_Undeserving.mp4 2019-04-16 22:20 217M
[[ ]] Joseph_Prince_-_His_Promises_Are_Yours_To_Own.mp4 2019-04-16 22:23 524M
[[ ]] Joseph_Prince_-_His_Radiance_Upon_You_Brings_Favor.mp4 2019-04-16 22:00 689M
[[ ]] Joseph_Prince_-_His_Resurrection,_Proof_Of_Your_Righteousness.mp4 2019-04-19 11:03 809M
[[ ]] Joseph_Prince_-_How_To_Be_Blessed_God's_Way.mp4 2019-04-16 21:08 776M
[[ ]] Joseph_Prince_-_How_To_Live_Free_From_The_Curse.mp4 2019-04-16 22:20 614M
[[ ]] Joseph_Prince_-_How_To_Make_Spirit-Led_Decisions.mp4 2019-04-16 21:14 871M
[[ ]] Joseph_Prince_-_How_To_Pray_When_You_Have_No_Prayer.mp4 2019-04-16 22:31 372M
[[ ]] Joseph_Prince_-_How_You_See_Jesus_Is_How_You_Will_Receive.mp4 2019-04-16 21:50 657M
[[ ]] Joseph_Prince_-_Immanuel__What_It_Means_To_Have_The_Lord_With_You.mp4 2019-04-16 21:30 210M
[[ ]] Joseph_Prince_-_Inherit_God's_Promises_By_Faith,_Not_by_Works.mp4 2019-04-16 21:49 521M
[[ ]] Joseph_Prince_-_Jesus_Draws_Near_When_You_Are_Discouraged_(Live_In_Israel).mp4 2019-04-16 21:18 561M
[[ ]] Joseph_Prince_-_Jesus_Our_Jubilee.mp4 2019-04-16 21:53 694M
[[ ]] Joseph_Prince_-_Jesus__Your_Reason_For_A_Fear-Free_Life.mp4 2019-04-16 21:05 210M
[[ ]] Joseph_Prince_-_Keys_To_Healing_In_The_Hebrew_Language.mp4 2019-04-16 21:00 250M
[[ ]] Joseph_Prince_-_Last_To_First_When_You_Trust_His_Goodness.mp4 2019-04-16 22:50 781M
[[ ]] Joseph_Prince_-_Learn_to_See_What_God_Sees.mp4 2019-04-16 21:46 419M
[[ ]] Joseph_Prince_-_Let_Go_And_Let_His_Supply_Flow.mp4 2019-04-16 22:33 461M
[[ ]] Joseph_Prince_-_Let_Go_and_Flow_in_the_Vine_Life.mp4 2019-04-16 22:26 458M
[[ ]] Joseph_Prince_-_Live_Bold_Without_Guilt_and_Fear.mp4 2019-04-16 22:04 427M
[[ ]] Joseph_Prince_-_Live_Confident.mp4 2019-04-16 21:03 426M
[[ ]] Joseph_Prince_-_Live_Life_Loved_By_The_Shepherd.mp4 2019-04-16 21:39 428M
[[ ]] Joseph_Prince_-_Live_Long,_Live_Strong.mp4 2019-04-16 22:35 621M
[[ ]] Joseph_Prince_-_Live_Strong_In_The_Father's_Love.mp4 2019-04-16 21:11 465M
[[ ]] Joseph_Prince_-_Live_Undefeated_In_Christ.mp4 2019-04-16 21:57 438M
[[ ]] Joseph_Prince_-_Make_Grace_Your_Way_of_Life.mp4 2019-04-16 22:15 520M
[[ ]] Joseph_Prince_-_Move_From_Predicament_To_Promotion.mp4 2019-04-16 22:46 813M
[[ ]] Joseph_Prince_-_No_Condemnation_Leads_to_Divine_Health.mp4 2019-04-16 21:59 685M
[[ ]] Joseph_Prince_-_Not_Ashamed_Of_The_Gospel.mp4 2019-04-16 22:22 235M
[[ ]] Joseph_Prince_-_Practical_Leadership_Keys_To_Living_Holy.mp4 2019-04-16 21:36 829M
[[ ]] Joseph_Prince_-_Receive_God's_Supply_for_All_of_Life's_Demands.mp4 2019-04-16 22:44 525M
[[ ]] Joseph_Prince_-_Redemption_Truths_That_Bless_Your_Relationships.mp4 2019-04-16 21:09 217M
[[ ]] Joseph_Prince_-_Rest_And_Receive_At_Jesus'_Feet.mp4 2019-04-16 21:54 449M
[[ ]] Joseph_Prince_-_Rest_In_Jesus'_Faith_For_Miracles.mp4 2019-04-16 21:04 519M
[[ ]] Joseph_Prince_-_Rest_Till_Your_Enemies_Become_Your_Footstool.mp4 2019-04-16 20:59 472M
[[ ]] Joseph_Prince_-_Rest!_God_Is_Working_Behind_The_Scenes.mp4 2019-04-16 22:43 523M
[[ ]] Joseph_Prince_-_Say_Amen_To_God's_Promises.mp4 2019-04-16 21:45 521M
[[ ]] Joseph_Prince_-_Set_Apart_To_Be_Kings_And_Priests.mp4 2019-04-16 21:34 802M
[[ ]] Joseph_Prince_-_Set_Free_to_Reign_in_Life.mp4 2019-04-16 22:06 528M
[[ ]] Joseph_Prince_-_Speak_Out_and_Find_Strength.mp4 2019-04-16 22:11 1.1G
[[ ]] Joseph_Prince_-_Speak_Out_by_Faith_and_Win.mp4 2019-04-16 21:20 412M
[[ ]] Joseph_Prince_-_Stay_on_Grace_Ground_and_Experience_True_Life.mp4 2019-04-16 21:41 1.0G
[[ ]] Joseph_Prince_-_Stronger_Through_Every_Trial_And_Battle.mp4 2019-04-16 22:47 789M
[[ ]] Joseph_Prince_-_The_Four_Gospels_Unlocked_for_Your_Blessings.mp4 2019-04-16 22:17 922M
[[ ]] Joseph_Prince_-_The_Friend_You_Can_Always_Depend_On.mp4 2019-04-16 22:40 1.1G
[[ ]] Joseph_Prince_-_The_God_Who_Goes_Before.mp4 2019-04-16 22:41 442M
[[ ]] Joseph_Prince_-_The_Health-Giving_Power_Of_A_Relaxed_Heart.mp4 2019-04-16 21:10 613M
[[ ]] Joseph_Prince_-_The_Heart_of_the_Father_Revealed.mp4 2019-04-16 21:44 468M
[[ ]] Joseph_Prince_-_The_Lord_Our_Righteousness.mp4 2019-04-16 21:02 423M
[[ ]] Joseph_Prince_-_The_Secret_of_Hearing_That_Brings_Untold_Blessings.mp4 2019-04-16 22:25 687M
[[ ]] Joseph_Prince_-_The_Spirit's_Rivers_Of_Provision_And_Healing.mp4 2019-04-16 21:52 404M
[[ ]] Joseph_Prince_-_Turn_Your_Frustrations_Into_Breakthroughs.mp4 2019-04-16 21:12 461M
[[ ]] Joseph_Prince_-_Unlocking_Redemption's_Blessings_In_Your_Life.mp4 2019-04-16 21:43 686M
[[ ]] Joseph_Prince_-_Victory_in_Your_Day_of_Trouble.mp4 2019-04-16 22:05 456M
[[ ]] Joseph_Prince_-_Walk_In_Constant_Victory_Over_Fear.mp4 2019-04-16 22:09 1.1G
[[ ]] Joseph_Prince_-_What_Is_Earnest_Prayer_To_God.mp4 2019-04-16 22:31 215M
[[ ]] Joseph_Prince_-_What_Makes_No_Weapon_Prosper_Against_You.mp4 2019-04-30 15:31 253M
[[ ]] Joseph_Prince_-_Where_Is_God_In_The_Midst_Of_Your_Trouble.mp4 2019-04-16 21:17 833M
[[ ]] Joseph_Prince_-_Win_Over_Discouragement,_Depression_and_Burnout.mp4 2019-04-16 22:24 396M
[[ ]] Joseph_Prince_-_Win_Over_Fear_and_Pride.mp4 2019-04-16 21:19 459M
[[ ]] Joseph_Prince_-_Win_Over_Guilt_and_Condemnation.mp4 2019-04-16 21:15 223M
[[ ]] Joseph_Prince_-_Wisdom_For_Holy_Living.mp4 2019-04-16 21:07 791M
[[ ]] Joseph_Prince_-_You_Are_Forgiven.mp4 2019-04-16 22:04 440M
[[ ]] Joseph_Prince_-_You_Can_Live_Healed.mp4 2019-04-16 22:33 217M
[[ ]] Joseph_Prince_-_Your_Blessed_Hope_In_Dark_Times.mp4 2019-04-16 22:21 464M
[[ ]] Joseph_Prince_-_Your_Only_Battle_is_Fight_to_Remain_at_Rest.mp4 2019-04-16 21:01 616M
[[ ]] Joseph_Prince_-_Your_Past_Does_Not_Determine_Your_Future.mp4 2019-04-16 21:22 685M
[[ ]] Joseph_Prince_-_Your_Reason_For_A_Fear_Free_Life.mp4 2019-04-16 21:13 468M
[[ ]] Joseph_Prince_-_Your_Security_in_Time_of_Shaking.mp4 2019-04-16 22:03 681M
=======================================================================================================================
EDIT1: Eu tentei isso não funciona porque os tamanhos de arquivo em Gigs não são adicionados:
$ curl -s $URL | html2text -width $COLUMNS | sed "s/M$//;s/G$/*1024/" | awk '/\.mp4/{size=$6;print"=> 6th field = "$6" size = "size >"/dev/stderr";total+=size}END{print "=> total = "total}'
=> 6th field = 428 size = 428
=> 6th field = 452 size = 452
=> 6th field = 391 size = 391
=> 6th field = 524 size = 524
=> 6th field = 894 size = 894
=> 6th field = 462 size = 462
=> 6th field = 388 size = 388
=> 6th field = 444 size = 444
=> 6th field = 705 size = 705
=> 6th field = 524 size = 524
=> 6th field = 453 size = 453
=> 6th field = 843 size = 843
=> 6th field = 609 size = 609
=> 6th field = 421 size = 421
=> 6th field = 189 size = 189
=> 6th field = 523 size = 523
=> 6th field = 209 size = 209
=> 6th field = 226 size = 226
=> 6th field = 609 size = 609
=> 6th field = 428 size = 428
=> 6th field = 224 size = 224
=> 6th field = 776 size = 776
=> 6th field = 441 size = 441
=> 6th field = 200 size = 200
=> 6th field = 776 size = 776
=> 6th field = 524 size = 524
=> 6th field = 732 size = 732
=> 6th field = 442 size = 442
=> 6th field = 225 size = 225
=> 6th field = 436 size = 436
=> 6th field = 459 size = 459
=> 6th field = 389 size = 389
=> 6th field = 217 size = 217
=> 6th field = 524 size = 524
=> 6th field = 689 size = 689
=> 6th field = 809 size = 809
=> 6th field = 776 size = 776
=> 6th field = 614 size = 614
=> 6th field = 871 size = 871
=> 6th field = 372 size = 372
=> 6th field = 657 size = 657
=> 6th field = 210 size = 210
=> 6th field = 521 size = 521
=> 6th field = 561 size = 561
=> 6th field = 694 size = 694
=> 6th field = 210 size = 210
=> 6th field = 250 size = 250
=> 6th field = 781 size = 781
=> 6th field = 419 size = 419
=> 6th field = 461 size = 461
=> 6th field = 458 size = 458
=> 6th field = 427 size = 427
=> 6th field = 426 size = 426
=> 6th field = 428 size = 428
=> 6th field = 621 size = 621
=> 6th field = 465 size = 465
=> 6th field = 438 size = 438
=> 6th field = 520 size = 520
=> 6th field = 813 size = 813
=> 6th field = 685 size = 685
=> 6th field = 235 size = 235
=> 6th field = 829 size = 829
=> 6th field = 525 size = 525
=> 6th field = 217 size = 217
=> 6th field = 449 size = 449
=> 6th field = 519 size = 519
=> 6th field = 472 size = 472
=> 6th field = 523 size = 523
=> 6th field = 521 size = 521
=> 6th field = 802 size = 802
=> 6th field = 528 size = 528
=> 6th field = 1.1*1024 size = 1.1*1024
=> 6th field = 412 size = 412
=> 6th field = 1.0*1024 size = 1.0*1024
=> 6th field = 789 size = 789
=> 6th field = 922 size = 922
=> 6th field = 1.1*1024 size = 1.1*1024
=> 6th field = 442 size = 442
=> 6th field = 613 size = 613
=> 6th field = 468 size = 468
=> 6th field = 423 size = 423
=> 6th field = 687 size = 687
=> 6th field = 404 size = 404
=> 6th field = 461 size = 461
=> 6th field = 686 size = 686
=> 6th field = 456 size = 456
=> 6th field = 1.1*1024 size = 1.1*1024
=> 6th field = 215 size = 215
=> 6th field = 450 size = 450
=> 6th field = 833 size = 833
=> 6th field = 396 size = 396
=> 6th field = 459 size = 459
=> 6th field = 223 size = 223
=> 6th field = 791 size = 791
=> 6th field = 440 size = 440
=> 6th field = 217 size = 217
=> 6th field = 464 size = 464
=> 6th field = 616 size = 616
=> 6th field = 685 size = 685
=> 6th field = 468 size = 468
=> 6th field = 681 size = 681
=> total = 49588.3
EDIT2: Eu finalmente fiz isso em Perl:
$ curl -s $URL | html2text -width $COLUMNS | perl -n -ale 'if(/mp4 /){$F[-1] =~ s/M$//;$F[-1] =~ s/G$/*1024/;$total+=eval $F[-1]}END{print "=> total = ",$total}'
=> total = 53987.2
EDIT3: Eu também tentei uma combinação de pupe jq, mas parece que data e tamanho não estão na matriz, portanto, não é fácil analisar:
$ curl -s $URL | pup 'json{}' | jq '.[] | .children[0].children[1].children[1].text'
"- 2019-04-16 22:32 428M 2019-04-16 22:36 452M 2019-04-16 21:48 391M 2019-04-16 21:27 524M 2019-04-16 22:28 894M 2019-04-30 16:00 761M 2019-04-16 22:48 462M 2019-04-16 22:12 388M 2019-04-16 22:16 444M 2019-04-16 22:38 705M 2019-04-16 21:28 524M 2019-04-16 22:18 453M 2019-04-25 13:21 804M 2019-04-16 21:23 843M 2019-04-16 21:38 609M 2019-04-16 22:37 421M 2019-04-16 21:41 189M 2019-04-16 22:13 523M 2019-04-16 22:20 209M 2019-04-16 21:57 226M 2019-04-16 22:14 609M 2019-04-16 21:00 428M 2019-04-16 22:42 224M 2019-04-16 21:30 776M 2019-04-16 21:29 441M 2019-04-16 21:56 200M 2019-04-16 21:56 776M 2019-04-16 21:25 524M 2019-04-16 22:30 732M 2019-04-16 21:47 442M 2019-04-16 21:51 225M 2019-04-16 21:25 436M 2019-04-16 21:31 459M 2019-04-16 21:32 389M 2019-04-16 22:20 217M 2019-04-16 22:23 524M 2019-04-16 22:00 689M 2019-04-19 11:03 809M 2019-04-16 21:08 776M 2019-04-16 22:20 614M 2019-04-16 21:14 871M 2019-04-16 22:31 372M 2019-04-16 21:50 657M 2019-04-16 21:30 210M 2019-04-16 21:49 521M 2019-04-16 21:18 561M 2019-04-16 21:53 694M 2019-04-16 21:05 210M 2019-04-16 21:00 250M 2019-04-16 22:50 781M 2019-04-16 21:46 419M 2019-04-16 22:33 461M 2019-04-16 22:26 458M 2019-04-16 22:04 427M 2019-04-16 21:03 426M 2019-04-16 21:39 428M 2019-04-16 22:35 621M 2019-04-16 21:11 465M 2019-04-16 21:57 438M 2019-04-16 22:15 520M 2019-04-16 22:46 813M 2019-04-16 21:59 685M 2019-04-16 22:22 235M 2019-04-16 21:36 829M 2019-04-16 22:44 525M 2019-04-16 21:09 217M 2019-04-16 21:54 449M 2019-04-16 21:04 519M 2019-04-16 20:59 472M 2019-04-16 22:43 523M 2019-04-16 21:45 521M 2019-04-16 21:34 802M 2019-04-16 22:06 528M 2019-04-16 22:11 1.1G 2019-04-16 21:20 412M 2019-04-16 21:41 1.0G 2019-04-16 22:47 789M 2019-04-16 22:17 922M 2019-04-16 22:40 1.1G 2019-04-16 22:41 442M 2019-04-16 21:10 613M 2019-04-16 21:44 468M 2019-04-16 21:02 423M 2019-04-16 22:25 687M 2019-04-16 21:52 404M 2019-04-16 21:12 461M 2019-04-16 21:43 686M 2019-04-16 22:05 456M 2019-04-16 22:09 1.1G 2019-04-16 22:31 215M 2019-04-30 15:31 253M 2019-04-16 21:17 833M 2019-04-16 22:24 396M 2019-04-16 21:19 459M 2019-04-16 21:15 223M 2019-04-16 21:07 791M 2019-04-16 22:04 440M 2019-04-16 22:33 217M 2019-04-16 22:21 464M 2019-04-16 21:01 616M 2019-04-16 21:22 685M 2019-04-16 21:13 468M 2019-04-16 22:03 681M"
Não existe uma maneira mais fácil ou "mais limpa"?
Use
numfmt --from=...para converter os tamanhos dos arquivos em bytes, então você pode resumi-los facilmente:| numfmt --to=iecpara obter tamanhos legíveis por humanos novamente, mais se você quiser--format '%.1f'obter casas decimais.--from=sipara tamanhos de arquivo baseados em 1.000.LC_ALL=C, mas recebo um erro com1.1G.Tente isso,
Gtalvez você possa substituir o NF por 6, pois você o menciona como um 6º campo em vez do último