KIF：如何自动运行/压力测试iOS应用程序以查找罕见UI错误的原因？

时间：2013-06-21 07:02:46

标签： objective-c debugging automated-tests race-condition kif

注意：我为搜索索引目的添加了 kif 标题，考虑到大部分答案都是讨论它

我正在寻找类似于selenium for iOS的东西，基本上是一个测试自动化/单元测试框架，它可以运行某个UI场景很多次，直到它崩溃，这将帮助我缩小UI错误的原因很少和随机发生。

（顺便说一下，我已经NSLogged了数据源/表交互的每一行代码，花了几个小时分析潜在的原因......但没有找到结论......再次发现这个bug很少发生。）

我看了一些unit testing frameworks in iOS，但它们似乎太多了。我不确定要选哪个。此外，我对selenium的引用基于猜想，因为我曾与那些过去曾在大型网络项目中使用Selenium的QA人员合作过（我假设iOS必须有类似的东西）。

现在我是一个从事iOS项目的单人团队，我将不得不戴上QA帽子来解决这个问题。

当遇到UITableView中插入的实际行数与数据源委托返回的行数之间存在差异时，我遇到了一个经典错误。这是错误消息：

*** Assertion failure in -[UITableView
 _endCellAnimationsWithContext:] Exception in insertRows: Invalid
 update: invalid number of rows in section 0.

The number of rows contained in an existing section after the update (2) must be equal to
 the number of rows contained in that section before the update (2),
 plus or minus the number of rows inserted or deleted from that section
 (1 inserted, 0 deleted) and plus or minus the number of rows moved
 into or out of that section (0 moved in, 0 moved out).

我点击UITableViewCell，将我带入另一个UITableView。有时它有效

enter image description here

有时（很少）它没有（带有上述错误）：

enter image description here

1 个答案:

答案 0 :(得分：3)

更新 ..我在分频器底部添加了关于KIF 2.0的示例代码..对于那些对KIF比我面临的具体问题更感兴趣的人：

经过一些研究和实验......我已经缩小了我对两个测试自动化库的选择范围： Frank和KIF。我最终决定使用KIF同时借用cucumber's Gherkin syntax来描述我的单元测试。

我选择KIF（而不是Frank）的原因是KIF基于100％obj-c，而不是使用ruby，{{1 }}。因此设置更简单，它更适用于我的狭窄测试用例要求。话虽如此，如果我的应用程序更复杂（即使用来自多个服务器的输入等），我承认Frank会更有用。您可以查看此excellent演示文稿的最后一个季度，以了解有关KIF，Frank和其他自动化测试框架（包括Apple自己的UI Automation）的优缺点的更多信息。

使用KIF后，我发现导致上述错误的错误，我可以100％的时间使用KIF重现它！它之所以很少发生的原因是因为它只是在我快速浏览屏幕时发生的......而且由于KIF自动执行了这些步骤......它以极快的速度完成它们......这暴露了这个bug :)。

以下是我用于测试的代码示例..这只是为了让您快速了解KIF（和Gherkin）可以为您做些什么：

在一个文件中我指定了我想要运行的场景：

Frank

每个场景映射到步骤（为了更多地了解小黄瓜语法 - 以及基于测试驱动程序开发的行为驱动开发，我强烈建议您阅读这本关于cucumber的优秀书籍）：

- (void)initializeScenarios;
{
    [self addScenario:[KIFTestScenario scenarioToCompleteSignInAndLoadInbox]];
    [self addScenario:[KIFTestScenario scenarioToFillAttachmentsWithData]];
    [self addScenario:[KIFTestScenario scenarioToViewAndLoadFileBucket]];
    [self addScenario:[KIFTestScenario scenarioToViewAndLoadFileBucketSubView]];
}

并使用KIF的UI自动化方法定义步骤（这只是一个例子）：

/* @given the application is at a fresh state
   @and   the user already has an imap email account with a valid username/pwd

   @then  the user can successfully log in
   @and   the inbox view will be loaded
   @and   the inbox will get loaded with the latest batch of emails in the user inbox
 */
+ (id)scenarioToCompleteSignInAndLoadInbox
{
    KIFTestScenario *scenario = 
      [KIFTestScenario scenarioWithDescription:@"Test that a user 
                                                 can successfully log in."];
    [scenario addStepsFromArray:[KIFTestStep stepsCompleteSignInAndLoadInbox]];

    return scenario;
}


/* @given that the user is already signed in
   @and   the user has already downloaded their folders 

   @then  the user can click on the folders view
   @and   the user can click on the 'attachments' remote folder
   @and   the latest batch from the 'attachments' remote folder will download
 */
+ (id)scenarioToFillAttachmentsWithData {
    KIFTestScenario* scenario = 
      [KIFTestScenario scenarioWithDescription:@"Test that we can view the 
                                                 attachments folder and fill 
                                                 it with data."];
    [scenario addStepsFromArray:[KIFTestStep stepsToFillAttachmentsWithData]];
    return scenario;

}

/* @given that the user is already signed in
   @and   the user has already downloaded their folders
   @and   the user has already downloaded attachments

   @then  the user can click on inbox menu button
   @and   the user can click on folder list menu button
   @and   the user can click on the file bucket icon (on the account list view)
   @and   the data for the file bucket is fetched from the dbase
   @and   the file bucket view displayes the attachments
 */
+ (id)scenarioToViewAndLoadFileBucket {
    KIFTestScenario *scenario = 
       [KIFTestScenario scenarioWithDescription:@"Test that a user can successfully 
                                                  view and load 
                                                  file bucket parent view"];
    [scenario addStepsFromArray:[KIFTestStep stepsToViewAndLoadFileBucketPage]];

    return scenario;
}

/* @given that the user is already signed in
   @and   the user has already downloaded their folders
   @and   the user has already downloaded attachments
   @and   the user has already opened file bucket view 

   @then  the user can click on a random row in the file bucket view table
   @and   the subview will retrieve data from the dbase pertaining to that row
   @and   the subview will display the data in the uitableview
 */
+ (id)scenarioToViewAndLoadFileBucketSubView {
    KIFTestScenario *scenario = 
       [KIFTestScenario scenarioWithDescription:@"Test that a user can successfully
                                                  view and load filet
                                                  bucket sub view"];
    [scenario addStepsFromArray:[KIFTestStep stepsToViewAndLoadFileBucketSubPage]];
    return scenario;   
}

KIF 2.0示例代码： KIF 2.0使用Xcode 5的全新test navigator ..这是一个巨大的改进，而不是KIF 1.0正在做的...现在你的测试感觉比过去更有机和自然..（即它实时进行..而不是创建未来运行的场景等等。）你甚至可以用播放按钮等测试每一个..你应该尝试一下。

这里有一些例子（再次使用gherkin语法）：

// this step assumes there is an attachment folder that contains emails with attachments
+ (NSArray *)stepsToFillAttachmentsWithData {

    NSMutableArray* steps = [@[] mutableCopy];

    [steps addObject:
        [KIFTestStep stepToTapViewWithAccessibilityLabel:@"InboxMenuButton"]];

    NSIndexPath* indexPath = 
        [NSIndexPath indexPathForRow:remoteAttachmentFolderNumber inSection:0];
    KIFTestStep* tapAttachmentRowStep = 
        [KIFTestStep stepToTapRowInTableViewWithAccessibilityLabel:
                                     @"attachments" atIndexPath:indexPath];

    [steps addObject:[KIFTestStep stepToWaitForNotificationName:
         (NSString *)kBeganSyncingOlderEmails object:nil           
                          whileExecutingStep:tapAttachmentRowStep]];

    [steps addObject:tapAttachmentRowStep];

    [steps addObject:
        [KIFTestStep stepToWaitForViewWithAccessibilityLabel:@"attachments"]];

    KIFTestStep *fillingInboxStep = 
        [KIFTestStep stepToWaitForNotificationName:
                                 (NSString *)kOldMailBatchDelivered object:nil];

    [fillingInboxStep setTimeout:kSpecialTimeoutForLongTests];
    [steps addObject:fillingInboxStep];

    return steps;
}

这里是类别文件中一些测试用例的实现：

#import <KIF/KIF.h>
#import "KIFUITestActor+EXAdditions.h"
#import "KIFUITestActor+UserRegistration.h"

@interface LoginTests : KIFTestCase

@end
@implementation LoginTests

- (void)testReset {
    [tester flushDbase];
    [tester reset];
}

/* @given that the app is in a fresh clean state
 @and   that no one has ever registered with the server

 @then  the user can register their themselves with the server
 @and   immediately start with the rider's map
 @and   their location on the map shows
 */

- (void)testRegistration
{
    [tester flushDbase];
    [tester reset];
    [tester singleUserRegistration];
    [tester showUserCurrentLocationOnMap];
}

/* @given that the user has already registered with the server
   @and the user is not currently logged in

 @then  the user can login using their user name and password
 @and   immediately start with the rider's map
 @and   their location on the map shows
 */
- (void)testSuccessfulLogin
{
    [tester reset];
    [tester login];
    [tester showUserCurrentLocationOnMap];
}

/* @given that the user has already registered 
   @and that the user is already logged in before app launch

 @then the user starts on the map view with the location visible
 @and the button prompts them to set pick up location
 */
- (void)testStartOfApplication {
    [tester showUserCurrentLocationOnMap];
    [tester showsPickUpButton];
}
@end
相关问题
最新问题