Rails优化复杂查询

时间:2012-04-05 15:20:49

标签: ruby-on-rails performance postgresql

这是后台,我有一个API方法,它将一个标准化的名称列表发送到我的服务器,然后将名称与给定的Persons对象匹配。每个人都有许多我想要返回的Post对象。我的数据库查询如下所示:

 @articles = Article.joins(:artists).select("artists.*").where({:artists=>{:name=>keys}}).group("#{Article.col_list}").order("articles.publish_date DESC").limit(25)

当名称列表很大(超过100)时,此查询在生产服务器上可能需要2秒钟。如何优化此查询以使其快速?

以下是查询的EXPLAIN。

 Limit  (cost=9144.19..9144.20 rows=25 width=802) (actual time=4341.409..4341.431 rows=25 loops=1)
   ->  Sort  (cost=9144.19..9150.74 rows=13096 width=802) (actual time=4341.408..4341.416 rows=25 loops=1)
         Sort Key: articles.publish_date
         Sort Method:  top-N heapsort  Memory: 47kB
         ->  HashAggregate  (cost=9030.99..9070.28 rows=13096 width=802) (actual time=4255.854..4315.125 rows=3916 loops=1)
               ->  Nested Loop  (cost=1581.27..8978.60 rows=13096 width=802) (actual time=7.443..3337.624 rows=39920 loops=1)
                     ->  Hash Join  (cost=1581.27..7650.04 rows=13096 width=4) (actual time=7.426..2627.426 rows=39920 loops=1)
                           Hash Cond: (articles_artists.artist_id = artists.id)
                           ->  Seq Scan on articles_artists  (cost=0.00..5476.38 rows=737461 width=8) (actual time=0.019..1310.806 rows=737515 loops=1)
                           ->  Hash  (cost=1579.83..1579.83 rows=413 width=4) (actual time=7.383..7.383 rows=300 loops=1)
                                 Buckets: 1024  Batches: 1  Memory Usage: 8kB
                                 ->  Bitmap Heap Scan on artists  (cost=477.60..1579.83 rows=413 width=4) (actual time=4.959..7.152 rows=300 loops=1)
                                       Recheck Cond: ((name)::text = ANY ('{"A BOY NAMED EARTH","A ROCKET TO MOON","A SKYLIT DRIVE","A VISIBLE BOY","AARON GILLESPIE",AB,"ABBIE BARRETT",ABERFELDY,"ABRA NIIRE","ABSOLUTE ZERO",ACDC,"ADAM GREEN",ADELE,ADO,"ADRINA THORPE",ADVENTURE,AEROSMITH,"AESOP ROCK",AFI,AIR,"AIR TRAFFIC",AKON,"AL GREEN","ALANIS MORISSETTE",ALCENDOR,"ALEX CHILTON","ALEXI MURDOCH","ALIAS EHREN","ALICE RUSSELL","ALICIA KEYS","AMANDA PALMER","AMERICAN FOOTBALL","AMY MACDONALD","AMY WINEHOUSE","ANALOG REBELLION",ANBERLIN,"ANDREW BIRD","ANDREW HILL","ANGELS AIRWAVES",AQUEDUCT,"ARCADE FIRE","ARETHA FRANKLIN","ART BRUT","ART GARFUNKEL","ARTIC MONKEYS","ARTIC MONKEYS","AS TALL AS LIONS",ASHANTI,"ATLAS SOUND","ATTACK ATTACK!",AUTOLUX,"AVENUE D","AVRIL LAVIGNE","B JU",BACK2SQUARE1,"BAD ENGLISH",BALMORHEA,"BAND OF HORSES",BARCELONA,"BAXTER DURY","BEASTIE BOYS",BEATLES,BECK,"BELLE AND SEBASTIAN","BEN FOLDS FIVE","BEN MARTIN","BETTE MIDLER","BETTIE SERVEERT",BEYONCE,"BIFFY CLYRO","BILLY BRAGG","BILLY BRAGG AND BLOKES","BILLY JOEL","BILLY OCEAN",BIRDY,"BLACK EYED PEAS","BLACK KIDS",BLACKSTONE,"BLIND PILOT","BLINK 182","BLOC PARTY","BLOODHOUND GANG",BLUEBOY,BLUR,"BOB DYLAN","BOB MARLEY","BOB MARLEY WAILERS","BOMBAY BICYCLE CLUB","BON IVER","BONNIE TYLER",BONOBO,"BOX CAR RACER","BOYS LIKE GIRLS","BOYZ II MEN",BREAKBOT,"BRENDAN BENSON","BRETT ANDERSON",BRICOLAGE,"BRITNEY SPEARS",BRONCO,"BRUCE SPRINGSTEEN","BRUNO MARS",BUSH,"BUSTA RHYMES",BUSTED,CAESARS,"CAGE ELEPHANT",CAKE,CALLA,"CALVIN HARRIS",CAMRON,CAMRON,CAMP,CAMPING,"CANDIE PAYNE",CASHEW,"CASSANDRA WILSOM",CASSIDY,"CATE LE BON","CHARLIE FEATHERS","CHERRY GHOST",CHIC,CHICAGO,CHINGY,"CHRIS BROWN","CHRISTINA AGUILERA","CHRISTINA STURMER",CHUMBAWAMBA,CIRCLE,"CLARENCE CARTER",CLIPSE,"COBRA STARSHIP",COLDPLAY,COMMON,CONSPIRATORS,COOLIO,COPELAND,"COPY HAHO",CORNELIUS,CREED,CULTS,"CYNDI LAUPER","CYPRESS HILL","CAT EMPIRE",CURE,"DA LATA","DAFT PUNK",DALMINJO,DAMERO,"DARREN HANLON","DASHBOARD CONFESSIONAL",DATAROCK,"DAVID BOWIE","DAVID E SUGAR","DAVID GUETTA","DAVID KITT","DEATH CAB FOR CUTIE","DIANA KRALL",DIGITALISM,"DINOSAUR JR","DISCO INFERNO",DISCOVERY,"DIZZEE RASCAL",DMX,"DON BLACKMAN","DR DOG",DRAKE,DRUGSTORE,DIPLOMATS,DREAM,EAGLES,EDITORS,ELBOW,ELECTRELANE,"ELIZA DOOLITTLE","ELLE MILANO","ELLIE GOULDING","ELTON JOHN","ELVIS COSTELLO WITH BURT BACHARACH","ELVIS PRESLEY",EMBRACE,EMINEM,"EMMY GREAT","EMPIRE! EMPIRE!","ERIC CHURCH","ERIC CLAPTON","ERIN MCCARLEY",ERRORS,"EUGENE KELLY","EXPLOSIONS IN SKY",FABOLOUS,FAILURE,"FAITH EVANS","FAR EAST MOVEMENT",FAUNTS,FEIST,FELT,FIELDS,"FLEET FOXES","FLIGHT OF CONCHORDS","FLORENCE MACHINE",FOALS,"FOO FIGHTERS","FOSTER PEOPLE","FOUR TET","FRANK TURNER","FRANZ FEDINAND",FREE,"FLAMING LIPS",GORILLAZ,"GRATEFUL DEAD",IDLEWILD,"IMANI COPPOLA","IMOGEN HEAP",INCUBUS,INCUBUS,"IRON WINE","J DILLA","J COLE",JACK,"JACK JOHNSON",JACOBITES,"JAMES BLAKE","JAMES BLUNT","JAMES BONG","JAMES BROWN","JAMIE T","JAN DELAY","JANE CULLUM","JASON DERULO","JASON MORAN","JAY Z","JAY Z","JEFF WAYNE","JENNIFER LOPEZ","JENNY LEWIS",JEREMIH,"JIM JONES","JIMMY EAT WORLD","JOHN COLTRANE","JOHN LEGEND","JOHN MAYER","JOY DIVISION","JUELZ SANTANA","JUNIOR BOYS","JUSTIN TIMBERLAKE","KAISER CHEIFS",KANTE,"KANYE WEST",KARMA,"KATE NASH","KATE NASH","KATY PERRY",KESHA,KEANE,KEANE,"KELLY ROWLAND","KENNY G","KERI HILSON","KEVIN AYERS","KID CUDI","KID HARPOON","KID LOCO","KID ROCK","KINGS OF LEON",KILLERS,KOOKS,"LADY GAGA","LADY SOVEREIGN","LED ZEPPELIN","LEE DORSEY",LEMAR,LEMONGRASS,"LEROY HUTSON","LIL WAYNE","LILY ALLEN","LITTLE COMETS","LITTLE JOY","LL COOL J",LLOYD,"LLOYD BANKS","LONDON LOUNGE",LOST,LOVE,LUDACRIS,LUDIQUE,"LUPE FIASCO","LUTHER VANDROS","MACHINE HEAD","MACY GRAY",MADONNA,MAINO,"MANU CHAO","MAREN MONTAUK","MARIAH CAREY","MARK FRY","MARK MORRISON","MARK RONSON","MAROON 5","MARY J BLIGE","MASSIVE ATTACK","MAT KEARNEY","MATTHEW DEAR","MEAT LOAF",METALLICA,MGMT,"MICHAEL GIACCHINO","MICHAEL JACKSON","MILES DAVIS","MISSY ELLIOTT","MOBB DEEP","MODEST MOUSE","MODEST MOUSE",MONK,MOON,MORRISSEY,MOTZ,"MOVING HEARTS",MUSE,MUTEMATH,"MY WRITES","MORNING OF",NAS,"NE YO","NICKI MINAJ","NINE INCH NAILS","P DIDDY","PATRICK WATSON","PAUL MCCARTNEY","PEARL JAM","PEPE ALGILAR","PEPE WHITE","PETER GABRIEL",PHARRELL,"PHIL COLLINS",PHISH,PITBULL,PIXIES,POLICE,PRINCE,"R KELLY",RADIOHEAD,RAKIM,RANK,"RASCAL FLATTS","RAY CHARLES","RED HOT CHILI PEPPERS",REDNEX,REENO,"RELIENT K","RICK ROSS","RICKY NELSON",RIFT,RIHANNA,"ROBBIE WILLIAMS","ROD STEWART","RODNEY HUNTER",ROKOKO,"ROLLING STONES","ROSCOE DASH","ROY DAVIS JR","RYAN LESLIE","RYAN LESLIE",RZA,"SARAH MCLACHLAN","SARAH RUSSELL",SEAL,"SHANIA TWAIN",SHERWOOD,"SMASH MOUTH",SMITHS,"SNOOP DOGG","SOMETHING CORPORATE",SPOON,SPOON,"ST LUNATICS",STARS,"STAT QUO","STEELY DAN","STEVIE WONDER","STYLES P","SUNSET RUBDOWN",SUPERCHUNK,"SWIZZ BEATZ","SECRET HANDSHAKE",SMITHS,"T PAIN",TI,"TALIB KWELI","TAYLOR SWIFT","THROWING MUSES",TLC,"TOM JONES","TOM PETTY","TONY YAYO",TOOL,"TORI AMOS",TORTOISE,TRAIN,"TREY SONGZ","TRIBE CALLED QUEST",TUBBS,TYCHO,"WHITE STRIPES","WAKA FLOCKA FLAME",WALE,WILCO,"WILL SMITH",U2,USHER,"YEAH YEAH YEAHS","YEAR OF RABBIT","YOUNG BUCK","YOUNG JEEZY","ZAC BROWN BAND",2PAC,"50 CENT"}'::text[]))
                                       ->  Bitmap Index Scan on index_artists_on_name  (cost=0.00..477.58 rows=413 width=0) (actual time=4.897..4.897 rows=311 loops=1)
                                             Index Cond: ((name)::text = ANY ('{"A BOY NAMED EARTH","A ROCKET TO MOON","A SKYLIT DRIVE","A VISIBLE BOY","AARON GILLESPIE",AB,"ABBIE BARRETT",ABERFELDY,"ABRA NIIRE","ABSOLUTE ZERO",ACDC,"ADAM GREEN",ADELE,ADO,"ADRINA THORPE",ADVENTURE,AEROSMITH,"AESOP ROCK",AFI,AIR,"AIR TRAFFIC",AKON,"AL GREEN","ALANIS MORISSETTE",ALCENDOR,"ALEX CHILTON","ALEXI MURDOCH","ALIAS EHREN","ALICE RUSSELL","ALICIA KEYS","AMANDA PALMER","AMERICAN FOOTBALL","AMY MACDONALD","AMY WINEHOUSE","ANALOG REBELLION",ANBERLIN,"ANDREW BIRD","ANDREW HILL","ANGELS AIRWAVES",AQUEDUCT,"ARCADE FIRE","ARETHA FRANKLIN","ART BRUT","ART GARFUNKEL","ARTIC MONKEYS","ARTIC MONKEYS","AS TALL AS LIONS",ASHANTI,"ATLAS SOUND","ATTACK ATTACK!",AUTOLUX,"AVENUE D","AVRIL LAVIGNE","B JU",BACK2SQUARE1,"BAD ENGLISH",BALMORHEA,"BAND OF HORSES",BARCELONA,"BAXTER DURY","BEASTIE BOYS",BEATLES,BECK,"BELLE AND SEBASTIAN","BEN FOLDS FIVE","BEN MARTIN","BETTE MIDLER","BETTIE SERVEERT",BEYONCE,"BIFFY CLYRO","BILLY BRAGG","BILLY BRAGG AND BLOKES","BILLY JOEL","BILLY OCEAN",BIRDY,"BLACK EYED PEAS","BLACK KIDS",BLACKSTONE,"BLIND PILOT","BLINK 182","BLOC PARTY","BLOODHOUND GANG",BLUEBOY,BLUR,"BOB DYLAN","BOB MARLEY","BOB MARLEY WAILERS","BOMBAY BICYCLE CLUB","BON IVER","BONNIE TYLER",BONOBO,"BOX CAR RACER","BOYS LIKE GIRLS","BOYZ II MEN",BREAKBOT,"BRENDAN BENSON","BRETT ANDERSON",BRICOLAGE,"BRITNEY SPEARS",BRONCO,"BRUCE SPRINGSTEEN","BRUNO MARS",BUSH,"BUSTA RHYMES",BUSTED,CAESARS,"CAGE ELEPHANT",CAKE,CALLA,"CALVIN HARRIS",CAMRON,CAMRON,CAMP,CAMPING,"CANDIE PAYNE",CASHEW,"CASSANDRA WILSOM",CASSIDY,"CATE LE BON","CHARLIE FEATHERS","CHERRY GHOST",CHIC,CHICAGO,CHINGY,"CHRIS BROWN","CHRISTINA AGUILERA","CHRISTINA STURMER",CHUMBAWAMBA,CIRCLE,"CLARENCE CARTER",CLIPSE,"COBRA STARSHIP",COLDPLAY,COMMON,CONSPIRATORS,COOLIO,COPELAND,"COPY HAHO",CORNELIUS,CREED,CULTS,"CYNDI LAUPER","CYPRESS HILL","CAT EMPIRE",CURE,"DA LATA","DAFT PUNK",DALMINJO,DAMERO,"DARREN HANLON","DASHBOARD CONFESSIONAL",DATAROCK,"DAVID BOWIE","DAVID E SUGAR","DAVID GUETTA","DAVID KITT","DEATH CAB FOR CUTIE","DIANA KRALL",DIGITALISM,"DINOSAUR JR","DISCO INFERNO",DISCOVERY,"DIZZEE RASCAL",DMX,"DON BLACKMAN","DR DOG",DRAKE,DRUGSTORE,DIPLOMATS,DREAM,EAGLES,EDITORS,ELBOW,ELECTRELANE,"ELIZA DOOLITTLE","ELLE MILANO","ELLIE GOULDING","ELTON JOHN","ELVIS COSTELLO WITH BURT BACHARACH","ELVIS PRESLEY",EMBRACE,EMINEM,"EMMY GREAT","EMPIRE! EMPIRE!","ERIC CHURCH","ERIC CLAPTON","ERIN MCCARLEY",ERRORS,"EUGENE KELLY","EXPLOSIONS IN SKY",FABOLOUS,FAILURE,"FAITH EVANS","FAR EAST MOVEMENT",FAUNTS,FEIST,FELT,FIELDS,"FLEET FOXES","FLIGHT OF CONCHORDS","FLORENCE MACHINE",FOALS,"FOO FIGHTERS","FOSTER PEOPLE","FOUR TET","FRANK TURNER","FRANZ FEDINAND",FREE,"FLAMING LIPS",GORILLAZ,"GRATEFUL DEAD",IDLEWILD,"IMANI COPPOLA","IMOGEN HEAP",INCUBUS,INCUBUS,"IRON WINE","J DILLA","J COLE",JACK,"JACK JOHNSON",JACOBITES,"JAMES BLAKE","JAMES BLUNT","JAMES BONG","JAMES BROWN","JAMIE T","JAN DELAY","JANE CULLUM","JASON DERULO","JASON MORAN","JAY Z","JAY Z","JEFF WAYNE","JENNIFER LOPEZ","JENNY LEWIS",JEREMIH,"JIM JONES","JIMMY EAT WORLD","JOHN COLTRANE","JOHN LEGEND","JOHN MAYER","JOY DIVISION","JUELZ SANTANA","JUNIOR BOYS","JUSTIN TIMBERLAKE","KAISER CHEIFS",KANTE,"KANYE WEST",KARMA,"KATE NASH","KATE NASH","KATY PERRY",KESHA,KEANE,KEANE,"KELLY ROWLAND","KENNY G","KERI HILSON","KEVIN AYERS","KID CUDI","KID HARPOON","KID LOCO","KID ROCK","KINGS OF LEON",KILLERS,KOOKS,"LADY GAGA","LADY SOVEREIGN","LED ZEPPELIN","LEE DORSEY",LEMAR,LEMONGRASS,"LEROY HUTSON","LIL WAYNE","LILY ALLEN","LITTLE COMETS","LITTLE JOY","LL COOL J",LLOYD,"LLOYD BANKS","LONDON LOUNGE",LOST,LOVE,LUDACRIS,LUDIQUE,"LUPE FIASCO","LUTHER VANDROS","MACHINE HEAD","MACY GRAY",MADONNA,MAINO,"MANU CHAO","MAREN MONTAUK","MARIAH CAREY","MARK FRY","MARK MORRISON","MARK RONSON","MAROON 5","MARY J BLIGE","MASSIVE ATTACK","MAT KEARNEY","MATTHEW DEAR","MEAT LOAF",METALLICA,MGMT,"MICHAEL GIACCHINO","MICHAEL JACKSON","MILES DAVIS","MISSY ELLIOTT","MOBB DEEP","MODEST MOUSE","MODEST MOUSE",MONK,MOON,MORRISSEY,MOTZ,"MOVING HEARTS",MUSE,MUTEMATH,"MY WRITES","MORNING OF",NAS,"NE YO","NICKI MINAJ","NINE INCH NAILS","P DIDDY","PATRICK WATSON","PAUL MCCARTNEY","PEARL JAM","PEPE ALGILAR","PEPE WHITE","PETER GABRIEL",PHARRELL,"PHIL COLLINS",PHISH,PITBULL,PIXIES,POLICE,PRINCE,"R KELLY",RADIOHEAD,RAKIM,RANK,"RASCAL FLATTS","RAY CHARLES","RED HOT CHILI PEPPERS",REDNEX,REENO,"RELIENT K","RICK ROSS","RICKY NELSON",RIFT,RIHANNA,"ROBBIE WILLIAMS","ROD STEWART","RODNEY HUNTER",ROKOKO,"ROLLING STONES","ROSCOE DASH","ROY DAVIS JR","RYAN LESLIE","RYAN LESLIE",RZA,"SARAH MCLACHLAN","SARAH RUSSELL",SEAL,"SHANIA TWAIN",SHERWOOD,"SMASH MOUTH",SMITHS,"SNOOP DOGG","SOMETHING CORPORATE",SPOON,SPOON,"ST LUNATICS",STARS,"STAT QUO","STEELY DAN","STEVIE WONDER","STYLES P","SUNSET RUBDOWN",SUPERCHUNK,"SWIZZ BEATZ","SECRET HANDSHAKE",SMITHS,"T PAIN",TI,"TALIB KWELI","TAYLOR SWIFT","THROWING MUSES",TLC,"TOM JONES","TOM PETTY","TONY YAYO",TOOL,"TORI AMOS",TORTOISE,TRAIN,"TREY SONGZ","TRIBE CALLED QUEST",TUBBS,TYCHO,"WHITE STRIPES","WAKA FLOCKA FLAME",WALE,WILCO,"WILL SMITH",U2,USHER,"YEAH YEAH YEAHS","YEAR OF RABBIT","YOUNG BUCK","YOUNG JEEZY","ZAC BROWN BAND",2PAC,"50 CENT"}'::text[]))
                     ->  Index Scan using articles_pkey on articles  (cost=0.00..0.10 rows=1 width=802) (actual time=0.006..0.006 rows=1 loops=39920)
                           Index Cond: (articles.id = articles_artists.article_id)
 Total runtime: 4343.603 ms
(18 rows)

3 个答案:

答案 0 :(得分:1)

这个突出:

- >在articles_artists上进行Seq Scan(成本= 0.00..5476.38行= 737461宽度= 8)(实际时间= 0.019..1310.806行= 737515循环= 1)

如果您在查询上运行EXPLAIN并在任何地方看到“Seq Scan”这一术语,则表明您需要检查是否拥有所有正确的索引。检查articles_artists并确保您在article_idartist_id都有索引,假设您已将其命名为。

此外,在您的数据库上运行VACUUM ANALYZE,看看是否加快了速度。 Postgresql的查询优化器取决于VACUUM ANALYZE生成的统计信息,因此如果表在运行时没有显着变化,查询性能可能会受到影响。

最后,考虑打破该查询并获取ruby来收集数据。有时您最好不要让数据库足够智能,以便正确执行复杂查询,而是在客户端手动优化它。

答案 1 :(得分:0)

看起来你要抓住最近的25篇文章。所以我会使用一个子查询来选择那些最近的25篇文章并将结果与​​艺术家表连接起来。

为什么会更好?因为您的查询构建了一个临时结果,该结果在订购结果并获取最新结果之前将每个艺术家与所有文章连接起来。

不......我错了。我看到艺术家名字的地方条件,这意味着我的解决方案无法运作。

所以我回来看看能否提供帮助。如果您查看发布的解释数据中的成本,艺术家名称“过滤器”会使您的成本最高。另请注意,艺术家名称都出现在那里,TWICE暗示您在嵌套循环内不止一次地支付此操作。

因此,如果可以,请构建一个临时表,该表是通过过滤器传递的艺术家表。然后使用该表以我之前描述的方式执行文章查找,您的查询将更快。

答案 2 :(得分:0)

我看到两个问题。第一个是Michele上面提到的seq扫描。第二个是嵌套循环连接,可能是由于此时发生了太多记录。

我会根据搜索条件查看您可以进一步查询的内容,以便尽可能少地选择项目。

我的猜测是,索引不会产生很大的差异,但会有额外的搜索条件,特别是如果它们可以达到索引。