詹金斯动态锁定

时间:2013-10-22 11:50:17

标签: jenkins locking jenkins-plugins

我是一名自动化工程师,我使用Jenkins进行自动化测试。 我必须在多个平台上测试每个测试,因此构建可能具有这些参数;

操作系统(Windows 7,Windows 8,XP 64位,XP 32位等)

服务器(我们产品的服务器,版本x,版本y等)

产品版本(x,y等)

还有更多......

选择的操作系统确定将使用哪个VM(虚拟机)作为测试依据。

问题是,我有很多这样的测试,并且那些运行测试的人并不总是检查已经使用的VM,或者他们是否在另一个自动测试期间使用特定VM设置了自动测试。

我希望构建等到VM清除后才能使用。

我尝试使用Locks and Latches插件 - 更改插件以检查每个锁,如果它的名称出现在构建参数中,如果是,请检查它的值。因此,如果锁的名称是“OS Type”,并且构建具有参数“OS Type = Windows 7”,则意味着构建搜索锁定“Windows 7”以查看它是否空闲。

我成功完成了上述部分 - 但现在当我运行测试时,第一个测试构建了它的环境,其他测试等待它完成整个构建,甚至没有检查锁! 多亏了这一点,我甚至不知道我做了什么。

有人可以帮忙吗?有没有人这样做过? 我将在下面发布代码,但正如我所说,我不确定它是否按预期工作。 提前谢谢!

public class LockWrapper extends BuildWrapper implements ResourceActivity {
private List<LockWaitConfig> locks;

public LockWrapper(List<LockWaitConfig> locks) {
    for(LockWaitConfig lock : locks)
    {

    }
    this.locks = locks;
}

public List<LockWaitConfig> getLocks() {
    return locks;
}

public void setLocks(List<LockWaitConfig> locks) {
    this.locks = locks;
}

@Override
public Descriptor<BuildWrapper> getDescriptor() {
    return DESCRIPTOR;
}

@Extension
public static final DescriptorImpl DESCRIPTOR = new DescriptorImpl();

/**
 * @see ResourceActivity#getResourceList()
 */
public ResourceList getResourceList() {
    ResourceList resources = new ResourceList();
    for (LockWaitConfig lock : locks) {
        resources.w(new Resource(null, "dynamic-locks/" + lock.getName(), DESCRIPTOR.getWriteLockCount()));
    }
    return resources;
}

@Override
public Environment setUp(AbstractBuild abstractBuild, Launcher launcher, BuildListener buildListener) throws IOException, InterruptedException {
    final List<NamedReentrantLock> backups = new ArrayList<NamedReentrantLock>();
    List<LockWaitConfig> locks = new ArrayList<LockWaitConfig>(this.locks);

    // sort this list of locks so that we _always_ ask for the locks in order
    Collections.sort(locks, new Comparator<LockWaitConfig>() {
        public int compare(LockWaitConfig o1, LockWaitConfig o2) {
            return o1.getName().compareTo(o2.getName());
        }
    });

    // build the list of "real" locks
    for (LockWaitConfig lock : locks) {
        NamedReentrantLock backupLock;
        String varName = lock.getName();
        String temp = varName;
        if(abstractBuild.getBuildVariables().containsKey(varName))
        {
            temp = abstractBuild.getBuildVariables().get(varName).toString();
            buildListener.getLogger().println("Variable " + varName + " found, replacing it with the value '" + temp + "'");
        }
        do {
            backupLock = DESCRIPTOR.backupLocks.get(temp);
            if (backupLock == null) {
                DESCRIPTOR.backupLocks.putIfAbsent(temp, new NamedReentrantLock(temp));
            }
        } while (backupLock == null);
        backups.add(backupLock);
    }

    final StringBuilder locksToGet = new StringBuilder();
    CollectionUtils.forAllDo(backups, new Closure() {
        public void execute(Object input) {
            locksToGet.append(((NamedReentrantLock) input).getName()).append(", ");
        }
    });

    buildListener.getLogger().println("[Dynamic Locks] Locks to get: " + locksToGet.substring(0, locksToGet.length()-2));

    boolean haveAll = false;
    while (!haveAll) {
        haveAll = true;
        List<NamedReentrantLock> locked = new ArrayList<NamedReentrantLock>();

        DESCRIPTOR.lockingLock.lock();
        try {
            for (NamedReentrantLock lock : backups) {
                buildListener.getLogger().print("[Dynamic Locks] Trying to get " + lock.getName() + "... ");
                if (lock.tryLock()) {
                    buildListener.getLogger().println(" Success");
                    locked.add(lock);
                } else {
                    buildListener.getLogger().println(" Failed, releasing all locks");
                    haveAll = false;
                    break;
                }
            }
            if (!haveAll) {
                // release them all
                for (ReentrantLock lock : locked) {
                    lock.unlock();
                }
            }
        } finally {
            DESCRIPTOR.lockingLock.unlock();
        }

        if (!haveAll) {
            buildListener.getLogger().println("[Dynamic Locks] Could not get all the locks, sleeping for 1 minute...");
            TimeUnit.SECONDS.sleep(60);
        }
    }

    buildListener.getLogger().println("[Dynamic Locks] Have all the locks, build can start");

    return new Environment() {
        @Override
        public boolean tearDown(AbstractBuild abstractBuild, BuildListener buildListener) throws IOException, InterruptedException {
            buildListener.getLogger().println("[Dynamic Locks] Releasing all the locks");
            for (ReentrantLock lock : backups) {
                lock.unlock();
            }
            buildListener.getLogger().println("[Dynamic Locks] All the locks released");
            return super.tearDown(abstractBuild, buildListener);
        }
    };
}

public void makeBuildVariables(AbstractBuild build, Map<String,String> variables) {
    final StringBuilder names = new StringBuilder();
    for (LockWaitConfig lock : locks) {
        if (names.length() > 0) {
            names.append(',');
        }
        names.append(lock.getName());
    }
    variables.put("LOCKS", names.toString());
}

public String getDisplayName() {
    return DESCRIPTOR.getDisplayName();
}

public static final class DescriptorImpl extends Descriptor<BuildWrapper> {
    private List<LockConfig> locks;

    /**
     * Required to work around HUDSON-2450.
     */
    private transient ConcurrentMap<String, NamedReentrantLock> backupLocks =
            new ConcurrentHashMap<String, NamedReentrantLock>();

    /**
     * Used to guarantee exclusivity when a build tries to get all its locks.
     */
    private transient ReentrantLock lockingLock = new ReentrantLock();

    DescriptorImpl() {
        super(LockWrapper.class);
        load();
    }

    public String getDisplayName() {
        return "Locks";
    }


    @Override
    public BuildWrapper newInstance(StaplerRequest req, JSONObject formData) throws FormException {
        List<LockWaitConfig> locks = req.bindParametersToList(LockWaitConfig.class, "locks.locks.");
        return new LockWrapper(locks);
    }

    @Override
    public boolean configure(StaplerRequest req, JSONObject formData) throws FormException {
        req.bindParameters(this, "locks.");
        locks = req.bindParametersToList(LockConfig.class, "locks.lock.");
        save();
        return super.configure(req, formData);
    }

    @Override
    public synchronized void save() {
        // let's remove blank locks
        CollectionUtils.filter(getLocks(), new Predicate() {
            public boolean evaluate(Object object) {
                return StringUtils.isNotBlank(((LockConfig) object).getName());
            }
        });

        // now, we can safely sort remaining locks
        Collections.sort(this.locks, new Comparator<LockConfig>() {
            public int compare(LockConfig lock1, LockConfig lock2) {
                return lock1.getName().compareToIgnoreCase(lock2.getName());
            }
        });

        super.save();
    }

    public List<LockConfig> getLocks() {
        if (locks == null) {
            locks = new ArrayList<LockConfig>();
            // provide default if we have none
            locks.add(new LockConfig("(default)"));
        }
        return locks;
    }

    public void setLocks(List<LockConfig> locks) {
        this.locks = locks;
    }

    public LockConfig getLock(String name) {
        for (LockConfig host : locks) {
            if (name.equals(host.getName())) {
                return host;
            }
        }
        return null;
    }

    public String[] getLockNames() {
        getLocks();
        String[] result = new String[locks.size()];
        for (int i = 0; i < result.length; i++) {
            result[i] = locks.get(i).getName();
        }
        return result;
    }

    public void addLock(LockConfig hostConfig) {
        locks.add(hostConfig);
        save();
    }

    /**
     * There wass a bug in the ResourceList.isCollidingWith,
     * this method used to determine the hack workaround if the bug is not fixed, but now only needs to
     * return 1.
     */
    synchronized int getWriteLockCount() {
        return 1;
    }
}

public static final class LockConfig implements Serializable {
    private String name;
    private transient AbstractBuild owner = null;

    public LockConfig() {
    }

    @DataBoundConstructor
    public LockConfig(String name) {
        this.name = name;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        LockConfig that = (LockConfig) o;

        if (name != null ? !name.equals(that.name) : that.name != null) return false;

        return true;
    }

    @Override
    public int hashCode() {
        int result;
        result = (name != null ? name.hashCode() : 0);
        return result;
    }
}

public static final class LockWaitConfig implements Serializable {
    private String name;
    private transient LockConfig lock;

    public LockWaitConfig() {
    }

    @DataBoundConstructor
    public LockWaitConfig(String name) {
        this.name = name;
    }

    public LockConfig getLock() {
        if (lock == null && name != null && !"".equals(name)) {
            setLock(DESCRIPTOR.getLock(name));
        }
        return lock;
    }

    public void setLock(LockConfig lock) {
        this.lock = lock;
    }

    public String getName() {
        if (lock == null) {
            return name;
        }
        return name = lock.getName();
    }

    public void setName(String name) {
        setLock(DESCRIPTOR.getLock(this.name = name));
    }

}

/**
 * Extends {@code ReentrantLock} to add a {@link #name} attribute (mainly
 * for display purposes).
 */
public static final class NamedReentrantLock extends ReentrantLock {
    private String name;

    public NamedReentrantLock(String name) {
        this.name = name;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

private static final Logger LOGGER = Logger.getLogger(LockWrapper.class.getName());

}

我改变的基本上就是这个;

for (LockWaitConfig lock : locks) {
        NamedReentrantLock backupLock;
        String varName = lock.getName();
        String temp = varName;
        if(abstractBuild.getBuildVariables().containsKey(varName))
        {
            temp = abstractBuild.getBuildVariables().get(varName).toString();
            buildListener.getLogger().println("Variable " + varName + " found, replacing it with the value '" + temp + "'");
        }
        do {
            backupLock = DESCRIPTOR.backupLocks.get(temp);
            if (backupLock == null) {
                DESCRIPTOR.backupLocks.putIfAbsent(temp, new NamedReentrantLock(temp));
            }
        } while (backupLock == null);
        backups.add(backupLock);
    }

用另一个例子澄清(谢谢Peter Schuetze提出这个问题)

我正在尝试运行可能具有相同资源(测试环境)的不同作业

对于这个例子,我将有两个不同的工作;

作业A在我选择的任何VM上运行一些测试。

作业B在我选择的任何VM上运行其他测试。

如果我选择作业A在VM'Windows 7'上运行,而其他人尝试在作业A开始运行后在VM'Windows 7'上运行作业B,我希望作业B被阻止,直到作业A完成。

我可以拥有许多Job A和Job B变体,每个变体都可以在不同的VM上运行,但考虑到我的平台矩阵,处理它太多了。

如果我想避免使用Locks插件,测试列表将如下所示;

  • 作业A - Windows 7 - 服务器A
  • 作业A - Windows 7 - 服务器B
  • 作业A - Windows 8 - 服务器A
  • 作业A - Windows 8 - 服务器B
  • 作业A - Windows XP x64 - 服务器A
  • 作业A - Windows XP x64 - 服务器B
  • 作业A - Windows XP x86 - 服务器A
  • 作业A - Windows XP x86 - 服务器B
  • 作业B - Windows 7 - 服务器A
  • 作业B - Windows 7 - 服务器B
  • 作业B - Windows 8 - 服务器A
  • 作业B - Windows 8 - 服务器B
  • 作业B - Windows XP x64 - 服务器A
  • 作业B - Windows XP x64 - 服务器B
  • 作业B - Windows XP x86 - 服务器A
  • 作业B - Windows XP x86 - 服务器B

请考虑实际上我现在...大约有20个工作,每个工作使用或多或少相同的资源(测试环境,服务器等......)

现在我已经做到了,所以我的工作清单就是这样;

  • 作业A - $ OS_TYPE - $ SERVER - $ Variable - $ Another_Variable
  • 作业B - $ OS_TYPE - $ SERVER - $ Variable - $ Another_Variable

为了确保多个作业不会同时使用任何资源,我需要使用locks插件,我需要它来接受变量作为参数。

如果您有任何其他问题或需要澄清,请随时询问:)

1 个答案:

答案 0 :(得分:1)

回顾一下。您修改了日志并锁定了插件以接受动态标签。您现在尝试使用相同的作业运行不同的测试。

您是否将作业配置为同时运行?此设置允许同一作业的实例并行运行。


BTW,这听起来像是多配置项目的典型用例。